{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "6048293f",
      "metadata": {
        "id": "6048293f"
      },
      "source": [
        "# Annotate text data using Active Learning with Cleanlab"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "K85mUO5yxhvq",
      "metadata": {
        "id": "K85mUO5yxhvq"
      },
      "source": [
        "Authored by: [Aravind Putrevu](https://huggingface.co/aravindputrevu)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0a242114",
      "metadata": {
        "id": "0a242114"
      },
      "source": [
        "In this notebook, I highlight the use of [active learning](https://arxiv.org/abs/2301.11856) to improve a fine-tuned Hugging Face Transformer for text classification, while keeping the total number of collected labels from human annotators low. When resource constraints prevent you from acquiring labels for the entirety of your data, active learning aims to save both time and money by selecting which examples data annotators should spend their effort labeling."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9a00d7ab",
      "metadata": {
        "id": "9a00d7ab"
      },
      "source": [
        "## What is Active Learning?\n",
        "\n",
        "Active Learning helps prioritize what data to label in order to maximize the performance of a supervised machine learning model trained on the labeled data. This process usually happens iteratively — at each round, active learning tells us which examples we should collect additional annotations for to improve our current model the most under a limited labeling budget. [ActiveLab](https://arxiv.org/abs/2301.11856) is an active learning algorithm that is particularly useful when the labels coming from human annotators are noisy and when we should collect one more annotation for a previously annotated example (whose label seems suspect) vs. for a not-yet-annotated example.  After collecting these new annotations for a batch of data to increase our training dataset, we re-train our model and evaluate its test accuracy.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "WJu_5YBqWgvr",
      "metadata": {
        "id": "WJu_5YBqWgvr"
      },
      "source": [
        "![ActiveLab thumb.webp]()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "WbDCLwmzWfsq",
      "metadata": {
        "id": "WbDCLwmzWfsq"
      },
      "source": [
        "In this notebook, I consider a binary text classification task: predicting whether a specific phrase is polite or impolite.\n",
        "\n",
        "Active learning with ActiveLab is much better than random selection when it comes to collecting additional annotations for Transformer models. It consistently produces much better models with approximately 50% less error rate, regardless of the total labeling budget.\n",
        "\n",
        "The rest of this notebook walks through the open-source code you can use to achieve these results."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6a3f7bfc",
      "metadata": {
        "id": "6a3f7bfc"
      },
      "source": [
        "## Setting up the environment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "QCRJsomHcbh5",
      "metadata": {
        "id": "QCRJsomHcbh5"
      },
      "outputs": [],
      "source": [
        "!pip install datasets==2.20.0 transformers==4.25.1 scikit-learn==1.1.2 matplotlib==3.5.3 cleanlab"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6668023c",
      "metadata": {
        "id": "6668023c"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "pd.set_option('max_colwidth', None)\n",
        "import numpy as np\n",
        "import random\n",
        "import transformers\n",
        "import datasets\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from cleanlab.multiannotator import get_majority_vote_label, get_active_learning_scores, get_label_quality_multiannotator\n",
        "from transformers import AutoTokenizer, AutoModel\n",
        "from transformers import AutoModelForSequenceClassification\n",
        "from transformers import TrainingArguments, Trainer\n",
        "from datasets import load_dataset, Dataset, DatasetDict, ClassLabel\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "from scipy.special import softmax\n",
        "from datetime import datetime"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c5c882d0",
      "metadata": {
        "id": "c5c882d0"
      },
      "source": [
        "## Collecting and Organizing Data\n",
        "\n",
        "Here we download the data that we need for this notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ba7e1270",
      "metadata": {},
      "outputs": [],
      "source": [
        "labeled_data_file = {\"labeled\": \"X_labeled_full.csv\"}\n",
        "unlabeled_data_file = {\"unlabeled\": \"X_labeled_full.csv\"}\n",
        "test_data_file = {\"test\": \"test.csv\"}\n",
        "\n",
        "X_labeled_full = load_dataset(\"Cleanlab/stanford-politeness\", split=\"labeled\", data_files=labeled_data_file)\n",
        "X_unlabeled = load_dataset(\"Cleanlab/stanford-politeness\", split=\"unlabeled\", data_files=unlabeled_data_file)\n",
        "test = load_dataset(\"Cleanlab/stanford-politeness\", split=\"test\", data_files=test_data_file)\n",
        "\n",
        "!wget -nc -O 'extra_annotations.npy' 'https://huggingface.co/datasets/Cleanlab/stanford-politeness/resolve/main/extra_annotations.npy?download=true'\n",
        "\n",
        "extra_annotations = np.load(\"extra_annotations.npy\",allow_pickle=True).item()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ca8eaf90",
      "metadata": {},
      "outputs": [],
      "source": [
        "X_labeled_full = X_labeled_full.to_pandas()\n",
        "X_labeled_full.set_index('id', inplace=True)\n",
        "X_unlabeled = X_unlabeled.to_pandas()\n",
        "X_unlabeled.set_index('id', inplace=True)\n",
        "test = test.to_pandas()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "818ec262",
      "metadata": {
        "id": "818ec262"
      },
      "source": [
        "## Classifying the Politeness of Text\n",
        "\n",
        "We are using [Stanford Politeness Corpus](https://convokit.cornell.edu/documentation/wiki_politeness.html) as the Dataset.\n",
        "\n",
        "It is structured as a binary text classification task, to classify whether each phrase is polite or impolite. Human annotators are given a selected text phrase and they provide an (imperfect) annotation regarding its politeness: **0** for impolite and **1** for polite.\n",
        "\n",
        "Training a Transformer classifier on the annotated data, we measure model accuracy over a set of held-out test examples, where I feel confident about their ground truth labels because they are derived from a consensus amongst 5 annotators who labeled each of these examples.\n",
        "\n",
        "As for the training data, we have:\n",
        "\n",
        "- `X_labeled_full`: our initial training set with just a small set of 100 text examples labeled with 2 annotations per example.\n",
        "- `X_unlabeled`: large set of 1900 unlabeled text examples we can consider having annotators label.\n",
        "- `extra_annotations`: pool of additional annotations we pull from when an annotation is requested for an example"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bfc644c1",
      "metadata": {
        "id": "bfc644c1"
      },
      "source": [
        "## Visualize Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4622088a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 874
        },
        "id": "4622088a",
        "outputId": "3fa0ca3a-9492-40f9-cd1f-21bc335b72c5"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "X_labeled_full"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-3a720cd1-6b98-47c0-be0c-0a8c328eff10\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>a6</th>\n",
              "      <th>a12</th>\n",
              "      <th>a16</th>\n",
              "      <th>a19</th>\n",
              "      <th>a20</th>\n",
              "      <th>a22</th>\n",
              "      <th>a39</th>\n",
              "      <th>a42</th>\n",
              "      <th>a52</th>\n",
              "      <th>...</th>\n",
              "      <th>a157</th>\n",
              "      <th>a158</th>\n",
              "      <th>a178</th>\n",
              "      <th>a180</th>\n",
              "      <th>a185</th>\n",
              "      <th>a193</th>\n",
              "      <th>a196</th>\n",
              "      <th>a197</th>\n",
              "      <th>a215</th>\n",
              "      <th>a216</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>450d326d</th>\n",
              "      <td>&lt;url&gt;. Congrats, or should I say good luck?</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6a22f4ec</th>\n",
              "      <td>Can I get some time to finish what I am doing without everything being deleted??</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>823f1104</th>\n",
              "      <td>Ok. Thank you for clarifying. Could you be more specific as to what you are specifying as \"the claim\" so that I may find relevant information to refute?</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7677905a</th>\n",
              "      <td>One wonders, of course, who \"Elliott of Macedon\" would have been. Probably something analogous to Brian of Nazareth but in a Macedonian phalanx?</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>a1ce799b</th>\n",
              "      <td>So, let me make sure I understand this. You think that, if we remove an image as it does not meet the NFCC, you would then be able to upload the same image, only this time, it would meet the NFCC?</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 34 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3a720cd1-6b98-47c0-be0c-0a8c328eff10')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-3a720cd1-6b98-47c0-be0c-0a8c328eff10 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-3a720cd1-6b98-47c0-be0c-0a8c328eff10');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-215ec3b9-a87d-48d3-88de-853f6322a1d0\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-215ec3b9-a87d-48d3-88de-853f6322a1d0')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-215ec3b9-a87d-48d3-88de-853f6322a1d0 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                                                                                                                                                                          text  \\\n",
              "id                                                                                                                                                                                                               \n",
              "450d326d                                                                                                                                                           <url>. Congrats, or should I say good luck?   \n",
              "6a22f4ec                                                                                                                      Can I get some time to finish what I am doing without everything being deleted??   \n",
              "823f1104                                              Ok. Thank you for clarifying. Could you be more specific as to what you are specifying as \"the claim\" so that I may find relevant information to refute?   \n",
              "7677905a                                                      One wonders, of course, who \"Elliott of Macedon\" would have been. Probably something analogous to Brian of Nazareth but in a Macedonian phalanx?   \n",
              "a1ce799b  So, let me make sure I understand this. You think that, if we remove an image as it does not meet the NFCC, you would then be able to upload the same image, only this time, it would meet the NFCC?   \n",
              "\n",
              "          a6  a12  a16  a19  a20  a22  a39  a42  a52  ...  a157  a158  a178  \\\n",
              "id                                                    ...                     \n",
              "450d326d NaN  NaN  1.0  NaN  NaN  0.0  NaN  NaN  NaN  ...   NaN   NaN   NaN   \n",
              "6a22f4ec NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  ...   NaN   NaN   NaN   \n",
              "823f1104 NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  ...   NaN   NaN   NaN   \n",
              "7677905a NaN  NaN  NaN  NaN  NaN  NaN  0.0  NaN  NaN  ...   NaN   NaN   NaN   \n",
              "a1ce799b NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  ...   NaN   NaN   NaN   \n",
              "\n",
              "          a180  a185  a193  a196  a197  a215  a216  \n",
              "id                                                  \n",
              "450d326d   NaN   NaN   NaN   NaN   NaN   NaN   NaN  \n",
              "6a22f4ec   NaN   NaN   NaN   NaN   NaN   NaN   NaN  \n",
              "823f1104   NaN   NaN   NaN   NaN   NaN   NaN   NaN  \n",
              "7677905a   NaN   NaN   0.0   NaN   NaN   NaN   NaN  \n",
              "a1ce799b   NaN   NaN   NaN   NaN   0.0   NaN   1.0  \n",
              "\n",
              "[5 rows x 34 columns]"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Multi-annotated Data\n",
        "X_labeled_full.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d9d7986e",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "d9d7986e",
        "outputId": "4699d001-2395-4fdd-9fff-edc92dabbab8"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"X_unlabeled\",\n  \"rows\": 1400,\n  \"fields\": [\n    {\n      \"column\": \"id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1400,\n        \"samples\": [\n          \"431df2fa\",\n          \"9f1f107e\",\n          \"b3d64e7e\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1398,\n        \"samples\": [\n          \"Just a question about the maintenance categories that FemtoBot automatically creates each month: I've noticed that the bot is frequently forced to recreate old, long-deleted monthly categories in the <person> queue, because an article got reverted to an old version for one reason or another and resulted in the old category being temporarily repopulated again. I'm just putting this forward as an idea for discussion, and am certainly not wedded to it u2014 but just to keep things simpler for everyone, I wonder what you would think about the feasibility and/or desirability of recoding the bot so that when it encounters a repopulated old maintenance category, it would retag the article to the ''current'' month instead of recreating the old one?\",\n          \"Thanks!  What type of documentation do you want u2014 an explanation of using the notes feature?\",\n          \"The benefit of a version of the sentence is that it helps clarify the differences between Lavoisier and Priestley; however, we need to be careful that we don't convey the wrong impression to the reader. How best to do this?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "X_unlabeled"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-50691c9d-9a9c-40bd-acde-bf5c4bfad34b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>486aff36</th>\n",
              "      <td>The review has been up there for something like six weeks, I notice. Think you'll be able to take care of those last couple of things?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>201d7655</th>\n",
              "      <td>How many other states follow the same pattern?  And do we really need it to?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>c9125774</th>\n",
              "      <td>You added the name Ken Taylor to the &lt;url&gt; page but there is no such person listed on the DOD website as having received that award. Who were you refering to?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>593ac8fb</th>\n",
              "      <td>I found &lt;url&gt; whilst looking for something else. Any use to you?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>d1fdcdba</th>\n",
              "      <td>If it were me I'd want to try and find out more about how/why this happened first before I continued to use that software. Have you asked at the talk page I mentioned above?</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-50691c9d-9a9c-40bd-acde-bf5c4bfad34b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-50691c9d-9a9c-40bd-acde-bf5c4bfad34b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-50691c9d-9a9c-40bd-acde-bf5c4bfad34b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-e6058ca7-68fc-49d4-8eaf-87d9b767fb73\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e6058ca7-68fc-49d4-8eaf-87d9b767fb73')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-e6058ca7-68fc-49d4-8eaf-87d9b767fb73 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                                                                                                                                                   text\n",
              "id                                                                                                                                                                                     \n",
              "486aff36                                         The review has been up there for something like six weeks, I notice. Think you'll be able to take care of those last couple of things?\n",
              "201d7655                                                                                                   How many other states follow the same pattern?  And do we really need it to?\n",
              "c9125774                 You added the name Ken Taylor to the <url> page but there is no such person listed on the DOD website as having received that award. Who were you refering to?\n",
              "593ac8fb                                                                                                               I found <url> whilst looking for something else. Any use to you?\n",
              "d1fdcdba  If it were me I'd want to try and find out more about how/why this happened first before I continued to use that software. Have you asked at the talk page I mentioned above?"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Unlabeled Data\n",
        "X_unlabeled.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84bbad84",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "84bbad84",
        "outputId": "d2eabdd6-1bd7-4c4b-df1e-8e9ad1f60473"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "<ipython-input-6-d9c8ad254414>:5: DeprecationWarning: Sampling from a set deprecated\n",
            "since Python 3.9 and will be removed in a subsequent version.\n",
            "  {k:extra_annotations[k] for k in random.sample(extra_annotations.keys(), 5)}\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "{'4235a537': {'a6': 0.0, 'a12': 0.0, 'a98': 0.0, 'a99': 0.0, 'a119': 0.0},\n",
              " '3d961d64': {'a68': 0.0, 'a70': 0.0, 'a79': 0.0, 'a99': 0.0, 'a199': 1.0},\n",
              " '4a5e75dc': {'a60': 1.0, 'a102': 1.0, 'a130': 1.0, 'a148': 1.0, 'a174': 1.0},\n",
              " '369a8b74': {'a65': 1.0, 'a68': 1.0, 'a71': 1.0, 'a157': 0.0, 'a161': 0.0},\n",
              " '356a4a74': {'a61': 0.0, 'a70': 1.0, 'a139': 0.0, 'a145': 1.0, 'a198': 1.0}}"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# extra_annotations contains the annotations that we will use when an additional annotation is requested.\n",
        "extra_annotations\n",
        "\n",
        "# Random sample of extra_annotations to see format.\n",
        "{k:extra_annotations[k] for k in random.sample(extra_annotations.keys(), 5)}"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "51a867fa",
      "metadata": {
        "id": "51a867fa"
      },
      "source": [
        "# View Some Examples From Test Set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "38f5974e",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        },
        "id": "38f5974e",
        "outputId": "9c5d0b24-49bd-4108-da3e-60d8e0d46872"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Impolite examples:\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"subset\",\n  \"rows\": 3,\n  \"fields\": [\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"And wasting our time as well. I can only repeat: why don't you do constructive work by adding contents about your beloved Makedonia?\",\n          \"Rather than tell me how wrong I was to close certain afd's maybe your time would be better spent dealing with the current afd backlog <url>. If my decisions were so wrong why haven't you re-opened them?\",\n          \"This was supposed to have been moved to <url> per the CFD. Why wasn't it moved?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "subset"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-41902311-0bdf-4503-8ea1-f7c599550be4\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>120</th>\n",
              "      <td>And wasting our time as well. I can only repeat: why don't you do constructive work by adding contents about your beloved Makedonia?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>150</th>\n",
              "      <td>Rather than tell me how wrong I was to close certain afd's maybe your time would be better spent dealing with the current afd backlog &lt;url&gt;. If my decisions were so wrong why haven't you re-opened them?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>326</th>\n",
              "      <td>This was supposed to have been moved to &lt;url&gt; per the CFD. Why wasn't it moved?</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-41902311-0bdf-4503-8ea1-f7c599550be4')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-41902311-0bdf-4503-8ea1-f7c599550be4 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-41902311-0bdf-4503-8ea1-f7c599550be4');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-8817f855-a31c-4b76-877a-f83b288c31b9\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8817f855-a31c-4b76-877a-f83b288c31b9')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-8817f855-a31c-4b76-877a-f83b288c31b9 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                                                                                                                                                                           text\n",
              "120                                                                        And wasting our time as well. I can only repeat: why don't you do constructive work by adding contents about your beloved Makedonia?\n",
              "150  Rather than tell me how wrong I was to close certain afd's maybe your time would be better spent dealing with the current afd backlog <url>. If my decisions were so wrong why haven't you re-opened them?\n",
              "326                                                                                                                             This was supposed to have been moved to <url> per the CFD. Why wasn't it moved?"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Polite examples:\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"subset\",\n  \"rows\": 3,\n  \"fields\": [\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"Hi there, I've raised the possibility of unprotecting the tamazepam page <url>. What are your thoughts?\",\n          \"Due to certain Edits the page alignment has changed. Could you please help?\",\n          \"I'm glad you're pleased with the general appearance.  Before I label all the streets, is the text size, font style, etc OK?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "subset"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-32540cf4-ec14-46c7-a81a-83647ef26abf\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>498</th>\n",
              "      <td>Hi there, I've raised the possibility of unprotecting the tamazepam page &lt;url&gt;. What are your thoughts?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>132</th>\n",
              "      <td>Due to certain Edits the page alignment has changed. Could you please help?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>131</th>\n",
              "      <td>I'm glad you're pleased with the general appearance.  Before I label all the streets, is the text size, font style, etc OK?</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-32540cf4-ec14-46c7-a81a-83647ef26abf')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-32540cf4-ec14-46c7-a81a-83647ef26abf button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-32540cf4-ec14-46c7-a81a-83647ef26abf');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-2c8907a1-dfff-4802-9411-e619d41560ee\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-2c8907a1-dfff-4802-9411-e619d41560ee')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-2c8907a1-dfff-4802-9411-e619d41560ee button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                                                                                            text\n",
              "498                      Hi there, I've raised the possibility of unprotecting the tamazepam page <url>. What are your thoughts?\n",
              "132                                                  Due to certain Edits the page alignment has changed. Could you please help?\n",
              "131  I'm glad you're pleased with the general appearance.  Before I label all the streets, is the text size, font style, etc OK?"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "num_to_label = {0:'Impolite', 1:\"Polite\"}\n",
        "for i in range(2):\n",
        "    print(f\"{num_to_label[i]} examples:\")\n",
        "    subset=test[test.label==i][['text']].sample(n=3, random_state=2)\n",
        "    print(subset)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c33e0753",
      "metadata": {},
      "source": [
        "Impolite Examples:\n",
        "\n",
        "|     |                                                                                                    text                                                                                                    |\n",
        "|----:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|\n",
        "| 120 |                                                                       And wasting our time as well. I can only repeat: why don't you do constructive work by adding contents about your beloved Makedonia? |\n",
        "| 150 | Rather than tell me how wrong I was to close certain afd's maybe your time would be better spent dealing with the current afd backlog <url>. If my decisions were so wrong why haven't you re-opened them? |\n",
        "| 326 |                                                                                                                            This was supposed to have been moved to <url> per the CFD. Why wasn't it moved? |"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "14b573eb",
      "metadata": {},
      "source": [
        "Polite Examples:\n",
        "\n",
        "|     |                                                            text                                                            |\n",
        "|----:|:--------------------------------------------------------------------------------------------------------------------------:|\n",
        "| 498 |                    Hi there, I've raised the possibility of unprotecting the tamazepam page <url>. What are your thoughts? |\n",
        "| 132 |                                                Due to certain Edits the page alignment has changed. Could you please help? |\n",
        "| 131 | I'm glad you're pleased with the general appearance. Before I label all the streets, is the text size, font style, etc OK? |"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5266e360",
      "metadata": {
        "id": "5266e360"
      },
      "source": [
        "# Helper Methods\n",
        "The following section contains all of the helper methods needed for this notebook.\n",
        "\n",
        "`get_idx_to_label` is designed for use in active learning scenarios, particularly when dealing with a mixture of labeled and unlabeled data. Its primary goal is to determine which examples (from both labeled and unlabeled datasets) should be selected for additional annotations based on their active learning scores. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "39f95b76",
      "metadata": {
        "id": "39f95b76"
      },
      "outputs": [],
      "source": [
        "# Helper method to get indices of examples with the lowest active learning score to collect more labels for.\n",
        "def get_idx_to_label(\n",
        "    X_labeled_full,\n",
        "    X_unlabeled,\n",
        "    extra_annotations,\n",
        "    batch_size_to_label,\n",
        "    active_learning_scores,\n",
        "    active_learning_scores_unlabeled=None,\n",
        "):\n",
        "    if active_learning_scores_unlabeled is None:\n",
        "        active_learning_scores_unlabeled = np.array([])\n",
        "\n",
        "    to_label_idx = []\n",
        "    to_label_idx_unlabeled = []\n",
        "\n",
        "    num_labeled = len(active_learning_scores)\n",
        "    active_learning_scores_combined = np.concatenate((active_learning_scores, active_learning_scores_unlabeled))\n",
        "    to_label_idx_combined = np.argsort(active_learning_scores_combined)\n",
        "\n",
        "    # We want to collect the n=batch_size best examples to collect another annotation for.\n",
        "    i = 0\n",
        "    while (len(to_label_idx)+len(to_label_idx_unlabeled)) < batch_size_to_label:\n",
        "        idx = to_label_idx_combined[i]\n",
        "        # We know this is an already annotated example.\n",
        "        if idx < num_labeled:\n",
        "            text_id = X_labeled_full.iloc[idx].name\n",
        "            # Make sure we have an annotation left to collect.\n",
        "            if text_id in extra_annotations and extra_annotations[text_id]:\n",
        "                to_label_idx.append(idx)\n",
        "        # We know this is an example that is currently not annotated.\n",
        "        else:\n",
        "            # Subtract off offset to get back original index.\n",
        "            idx -= num_labeled\n",
        "            text_id = X_unlabeled.iloc[idx].name\n",
        "            # Make sure we have an annotation left to collect.\n",
        "            if text_id in extra_annotations and extra_annotations[text_id]:\n",
        "                to_label_idx_unlabeled.append(idx)\n",
        "        i+=1\n",
        "\n",
        "    to_label_idx = np.array(to_label_idx)\n",
        "    to_label_idx_unlabeled = np.array(to_label_idx_unlabeled)\n",
        "    return to_label_idx, to_label_idx_unlabeled"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6a3527a4",
      "metadata": {},
      "source": [
        "`get_idx_to_label_random` is designed for an active learning context where the selection of data points for additional annotation is done randomly rather than based on a model's uncertainty or learning scores. This approach might be used as a baseline to compare against more sophisticated active learning strategies or in scenarios where it's unclear how to score examples."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c5f75d4a",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Helper method to get indices of random examples to collect more labels for.\n",
        "def get_idx_to_label_random(\n",
        "    X_labeled_full,\n",
        "    X_unlabeled,\n",
        "    extra_annotations,\n",
        "    batch_size_to_label\n",
        "):\n",
        "    to_label_idx = []\n",
        "    to_label_idx_unlabeled = []\n",
        "\n",
        "    # Generate list of indices for both sets of examples.\n",
        "    labeled_idx = [(x, 'labeled') for x in range(len(X_labeled_full))]\n",
        "    unlabeled_idx = []\n",
        "    if X_unlabeled is not None:\n",
        "        unlabeled_idx = [(x, 'unlabeled') for x in range(len(X_unlabeled))]\n",
        "    combined_idx = labeled_idx + unlabeled_idx\n",
        "\n",
        "    # We want to collect the n=batch_size random examples to collect another annotation for.\n",
        "    while (len(to_label_idx)+len(to_label_idx_unlabeled)) < batch_size_to_label:\n",
        "        # Random choice from indices.\n",
        "        # We time-seed to ensure randomness.\n",
        "        random.seed(datetime.now().timestamp())\n",
        "        choice = random.choice(combined_idx)\n",
        "        idx, which_subset = choice\n",
        "        # We know this is an already annotated example.\n",
        "        if which_subset == 'labeled':\n",
        "            text_id = X_labeled_full.iloc[idx].name\n",
        "            # Make sure we have an annotation left to collect.\n",
        "            if text_id in extra_annotations and extra_annotations[text_id]:\n",
        "                to_label_idx.append(idx)\n",
        "            combined_idx.remove(choice)\n",
        "        # We know this is an example that is currently not annotated.\n",
        "        else:\n",
        "            text_id = X_unlabeled.iloc[idx].name\n",
        "            # Make sure we have an annotation left to collect.\n",
        "            if text_id in extra_annotations and extra_annotations[text_id]:\n",
        "                to_label_idx_unlabeled.append(idx)\n",
        "            combined_idx.remove(choice)\n",
        "\n",
        "    to_label_idx = np.array(to_label_idx)\n",
        "    to_label_idx_unlabeled = np.array(to_label_idx_unlabeled)\n",
        "    return to_label_idx, to_label_idx_unlabeled"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7c2549a5",
      "metadata": {},
      "source": [
        "Below are some utility methods which helps us to compute standard deviation, selecting a specific annotator who has previously annotated the example, and some token functions to Tokenize text examples."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2c6cb493",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Helper method to compute std dev across 2D array of accuracies.\n",
        "def compute_std_dev(accuracy):\n",
        "    def compute_std_dev_ind(accs):\n",
        "        mean = np.mean(accs)\n",
        "        std_dev = np.std(accs)\n",
        "        return np.array([mean - std_dev, mean + std_dev])\n",
        "\n",
        "    std_dev = np.apply_along_axis(compute_std_dev_ind, 0, accuracy)\n",
        "    return std_dev\n",
        "\n",
        "# Helper method to select which annotator we should collect another annotation from.\n",
        "def choose_existing(annotators, existing_annotators):\n",
        "    for annotator in annotators:\n",
        "        # If we find one that has already given an annotation, we return it.\n",
        "        if annotator in existing_annotators:\n",
        "            return annotator\n",
        "    # If we don't find an existing, just return a random one.\n",
        "    choice = random.choice(list(annotators.keys()))\n",
        "    return choice\n",
        "\n",
        "# Helper method for Trainer.\n",
        "def compute_metrics(p):\n",
        "    logits, labels = p\n",
        "    pred = np.argmax(logits, axis=1)\n",
        "    pred_probs = softmax(logits, axis=1)\n",
        "    accuracy = accuracy_score(y_true=labels, y_pred=pred)\n",
        "    return {\"logits\":logits, \"pred_probs\":pred_probs, \"accuracy\": accuracy}\n",
        "\n",
        "# Helper method to tokenize text.\n",
        "def tokenize_function(examples):\n",
        "    model_name = \"distilbert-base-uncased\"\n",
        "    tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
        "    return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n",
        "\n",
        "# Helper method to tokenize given dataset.\n",
        "def tokenize_data(data):\n",
        "    dataset = Dataset.from_dict({\"label\":data['label'] , \"text\": data['text'].values})\n",
        "    tokenized_dataset = dataset.map(tokenize_function, batched=True)\n",
        "    tokenized_dataset = tokenized_dataset.cast_column(\"label\", ClassLabel(names = [\"0\",\"1\"]))\n",
        "    return tokenized_dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8bf51c47",
      "metadata": {},
      "source": [
        "`get_trainer` function here is designed to set up a training environment for a text classification task using DistilBERT, a distilled version of the BERT model that is lighter and faster."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9c286ef1",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Helper method to initiate a new Trainer with given train and test sets.\n",
        "def get_trainer(train_set, test_set):\n",
        "\n",
        "    # Model params.\n",
        "    model_name = \"distilbert-base-uncased\"\n",
        "    model_folder = \"model_training\"\n",
        "    max_training_steps = 300\n",
        "    num_classes = 2\n",
        "\n",
        "    # Set training args.\n",
        "    # We time-seed to ensure randomness between different benchmarking runs.\n",
        "    training_args = TrainingArguments(\n",
        "        max_steps=max_training_steps,\n",
        "        output_dir=model_folder,\n",
        "        seed = int(datetime.now().timestamp())\n",
        "    )\n",
        "\n",
        "    # Tokenize train/test set.\n",
        "    train_tokenized_dataset = tokenize_data(train_set)\n",
        "    test_tokenized_dataset = tokenize_data(test_set)\n",
        "\n",
        "    # Initiate a pre-trained model.\n",
        "    model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=num_classes)\n",
        "    trainer = Trainer(\n",
        "        model=model,\n",
        "        args=training_args,\n",
        "        compute_metrics = compute_metrics,\n",
        "        train_dataset = train_tokenized_dataset,\n",
        "        eval_dataset = test_tokenized_dataset,\n",
        "    )\n",
        "    return trainer"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b13d057e",
      "metadata": {},
      "source": [
        "`get_pred_probs` function performs out-of-sample prediction probability computation for a given dataset using cross-validation, with additional handling for unlabeled data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5e47a71d",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Helper method to manually compute cross-validated predicted probabilities needed for ActiveLab.\n",
        "def get_pred_probs(X, X_unlabeled):\n",
        "    \"\"\"Uses cross-validation to obtain out-of-sample predicted probabilities\n",
        "    for given dataset\"\"\"\n",
        "\n",
        "    # Generate cross-val splits.\n",
        "    n_splits = 3\n",
        "    skf = StratifiedKFold(n_splits=n_splits, shuffle=True)\n",
        "    skf_splits = [\n",
        "        [train_index, test_index]\n",
        "        for train_index, test_index in skf.split(X=X['text'], y=X['label'])\n",
        "    ]\n",
        "\n",
        "    # Initiate empty array to store pred_probs.\n",
        "    num_examples, num_classes = len(X), len(X.label.value_counts())\n",
        "    pred_probs = np.full((num_examples, num_classes), np.NaN)\n",
        "    pred_probs_unlabeled = None\n",
        "\n",
        "    # If we use up all examples from the initial unlabeled pool, X_unlabeled will be None.\n",
        "    if X_unlabeled is not None:\n",
        "        pred_probs_unlabeled = np.full((n_splits, len(X_unlabeled), num_classes), np.NaN)\n",
        "\n",
        "    # Iterate through cross-validation folds.\n",
        "    for split_num, split in enumerate(skf_splits):\n",
        "        train_index, test_index = split\n",
        "\n",
        "        train_set = X.iloc[train_index]\n",
        "        test_set = X.iloc[test_index]\n",
        "\n",
        "        # Get trainer with train/test subsets.\n",
        "        trainer = get_trainer(train_set, test_set)\n",
        "        trainer.train()\n",
        "        eval_metrics = trainer.evaluate()\n",
        "\n",
        "        # Get pred_probs and insert into dataframe.\n",
        "        pred_probs_fold = eval_metrics['eval_pred_probs']\n",
        "        pred_probs[test_index] = pred_probs_fold\n",
        "\n",
        "        # Since we don't have labels for the unlabeled pool, we compute pred_probs at each round of CV\n",
        "        # and then average the results at the end.\n",
        "        if X_unlabeled is not None:\n",
        "            dataset_unlabeled = Dataset.from_dict({\"text\": X_unlabeled['text'].values})\n",
        "            unlabeled_tokenized_dataset = dataset_unlabeled.map(tokenize_function, batched=True)\n",
        "            logits = trainer.predict(unlabeled_tokenized_dataset).predictions\n",
        "            curr_pred_probs_unlabeled = softmax(logits, axis=1)\n",
        "            pred_probs_unlabeled[split_num] = curr_pred_probs_unlabeled\n",
        "\n",
        "    # Here we average the pred_probs from each round of CV to get pred_probs for the unlabeled pool.\n",
        "    if X_unlabeled is not None:\n",
        "        pred_probs_unlabeled = np.mean(np.array(pred_probs_unlabeled), axis=0)\n",
        "\n",
        "    return pred_probs, pred_probs_unlabeled"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0b2a22ff",
      "metadata": {},
      "source": [
        "`get_annotator` function determines the most appropriate annotator to collect a new annotation from for a specific example, based on a set of criteria while `get_annotation` focused on collecting an actual annotation for a given example from a chosen annotator, it also deletes the collected annotation from the pool to prevent it from being selected again."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1d2d7936",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Helper method to determine which annotator to collect annotation from for given example.\n",
        "def get_annotator(example_id):\n",
        "    # Update who has already annotated atleast one example.\n",
        "    existing_annotators = set(X_labeled_full.drop('text', axis=1).columns)\n",
        "    # Returns the annotator we want to collect annotation from.\n",
        "    # Chooses existing annotators first.\n",
        "    annotators = extra_annotations[example_id]\n",
        "    chosen_annotator = choose_existing(annotators, existing_annotators)\n",
        "    return chosen_annotator\n",
        "\n",
        "# Helper method to collect an annotation for given text example.\n",
        "def get_annotation(example_id, chosen_annotator):\n",
        "\n",
        "    # Collect new annotation.\n",
        "    new_annotation = extra_annotations[example_id][chosen_annotator]\n",
        "\n",
        "    # Remove annotation.\n",
        "    del extra_annotations[example_id][chosen_annotator]\n",
        "\n",
        "    return new_annotation"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "eM_CNS3weUdt",
      "metadata": {
        "id": "eM_CNS3weUdt"
      },
      "source": [
        "Run the following cell to hide the HTML output from the next model training block."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "675c0365",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "675c0365",
        "outputId": "e310200c-dd64-4ebd-fd9f-f58736be085e"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style>\n",
              "    div.output_stderr {\n",
              "    display: none;\n",
              "    }\n",
              "</style>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "%%html\n",
        "<style>\n",
        "    div.output_stderr {\n",
        "    display: none;\n",
        "    }\n",
        "</style>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1c48c738",
      "metadata": {
        "id": "1c48c738"
      },
      "source": [
        "## Methodology Used\n",
        "\n",
        "For each **active learning** round we:\n",
        "\n",
        "1. Compute ActiveLab consensus labels for each training example derived from all annotations collected thus far.\n",
        "2. Train our Transformer classification model on the current training set using these consensus labels.\n",
        "3. Evaluate test accuracy on the test set (which has high-quality ground truth labels).\n",
        "4. Run cross-validation to get out-of-sample predicted class probabilities from our model for the entire training set and unlabeled set.\n",
        "5. Get ActiveLab active learning scores for each example in the training set and unlabeled set. These scores estimate how informative it would be to collect another annotation for each example.\n",
        "6. Select a subset (*n = batch_size*) of examples with the lowest active learning scores.\n",
        "7. Collect one additional annotation for each of the *n* selected examples.\n",
        "8. Add the new annotations (and new previously non-annotated examples if selected) to our training set for the next iteration.\n",
        "\n",
        "I subsequently compare models trained on data labeled via active learning vs. data labeled via **random selection**.  For each random selection round, I use majority vote consensus instead of ActiveLab consensus (in Step 1) and then just randomly select the **n** examples to collect an additional label for instead of using ActiveLab scores (in Step 6).\n",
        "\n",
        "More intuition on Activelab Consensus labels and Active learning scores are shared further in the notebook.  \n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "yQB8KpoWeblr",
      "metadata": {
        "id": "yQB8KpoWeblr"
      },
      "source": [
        "![activelab.png]()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2Fe89jH9eaCR",
      "metadata": {
        "id": "2Fe89jH9eaCR"
      },
      "source": [
        "### Model Training and Evaluation\n",
        "\n",
        "I first tokenize my test and train sets, and then initialize a pre-trained DistilBert Transformer model. Fine-tuning DistilBert with 300 training steps produced a good balance between accuracy and training time for my data. This classifier outputs predicted class probabilities which I convert to class predictions before evaluating their accuracy.\n",
        "\n",
        "### Use Active Learning Scores to Decide what to Label Next\n",
        "During each round of Active Learning, we fit our Transformer model via 3-fold cross-validation on the current training set. This allows us to get out-of-sample predicted class probabilities for each example in the training set and we can also use the trained Transformer to get out-of-sample predicted class probabilities for each example in the unlabeled pool. All of this is internally implemented in the `get_pred_probs` helper method. The use of out-of-sample predictions helps us avoid bias due to potential overfitting.\n",
        "\n",
        "Once I have these probabilistic predictions, I pass them into the `get_active_learning_scores` method from the open-source [cleanlab](https://github.com/cleanlab/cleanlab) package, which implements the [ActiveLab algorithm](https://arxiv.org/abs/2301.11856).  This method provides us with scores for all of our labeled and unlabeled data. Lower scores indicate data points for which collecting one additional label should be most informative for our current model (scores are directly comparable between labeled and unlabeled data).\n",
        "\n",
        "I form a batch of examples with the lowest scores as the examples to collect an annotation for (via the `get_idx_to_label` method). Here I always collect the exact same number of annotations in each round (under both the active learning and random selection approaches). For this application, I limit the maximum number of annotations per example to 5 (don’t want to spend effort labeling the same example over and over again).\n",
        "\n",
        "### Adding new Annotations\n",
        "The `combined_example_ids` are the ids of the text examples we want to collect an annotation for. For each of these, we use the `get_annotation` helper method to collect a new annotation from an annotator. Here, we prioritize selecting annotations from annotators who have already annotated another example. If none of the annotators for the given example exist in the training set, we randomly select one. In this case, we add a new column to our training set which represents the new annotator. Finally, we add the newly collected annotation to the training set. If the corresponding example was previously non-annotated, we also add it to the training set and remove it from the unlabeled collection.\n",
        "\n",
        "We’ve now completed one round of collecting new annotations and retrain the Transformer model on the updated training set.  We repeat this process in multiple rounds to keep growing the training dataset and improving our model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5f5cad58",
      "metadata": {
        "id": "5f5cad58"
      },
      "outputs": [],
      "source": [
        "# For this Active Learning demo, we add 25 additional annotations to the training set\n",
        "# each iteration, for 25 rounds.\n",
        "num_rounds = 25\n",
        "batch_size_to_label = 25\n",
        "model_accuracy_arr = np.full(num_rounds, np.nan)\n",
        "\n",
        "# The 'selection_method' varible determines if we use ActiveLab or random selection\n",
        "# to choose the new annotations each round.\n",
        "selection_method = 'random'\n",
        "# selection_method = 'active_learning'\n",
        "\n",
        "# Each round we:\n",
        "# - train our model\n",
        "# - evaluate on unchanging test set\n",
        "# - collect and add new annotations to training set\n",
        "for i in range(num_rounds):\n",
        "\n",
        "    # X_labeled_full is updated each iteration. We drop the text column which leaves us with just the annotations.\n",
        "    multiannotator_labels = X_labeled_full.drop(['text'], axis=1)\n",
        "\n",
        "    # Use majority vote when using random selection to select the consensus label for each example.\n",
        "    if i == 0 or selection_method == 'random':\n",
        "        consensus_labels = get_majority_vote_label(multiannotator_labels)\n",
        "\n",
        "    # When using ActiveLab, use cleanlab's CrowdLab to select the consensus label for each example.\n",
        "    else:\n",
        "        results = get_label_quality_multiannotator(\n",
        "            multiannotator_labels,\n",
        "            pred_probs_labeled,\n",
        "            calibrate_probs=True,\n",
        "        )\n",
        "        consensus_labels = results[\"label_quality\"][\"consensus_label\"].values\n",
        "\n",
        "    # We only need the text and label columns.\n",
        "    train_set = X_labeled_full[['text']]\n",
        "    train_set['label'] = consensus_labels\n",
        "    test_set = test[['text', 'label']]\n",
        "\n",
        "    # Train our Transformer model on the full set of labeled data to evaluate model accuracy for the current round.\n",
        "    # This is an optional step for demonstration purposes, in practical applications\n",
        "    # you may not have ground truth labels.\n",
        "    trainer = get_trainer(train_set, test_set)\n",
        "    trainer.train()\n",
        "    eval_metrics = trainer.evaluate()\n",
        "    # set statistics\n",
        "    model_accuracy_arr[i] = eval_metrics['eval_accuracy']\n",
        "\n",
        "    # For ActiveLab, we need to run cross-validation to get out-of-sample predicted probabilites.\n",
        "    if selection_method == 'active_learning':\n",
        "        pred_probs, pred_probs_unlabeled = get_pred_probs(train_set, X_unlabeled)\n",
        "\n",
        "        # Compute active learning scores.\n",
        "        active_learning_scores, active_learning_scores_unlabeled = get_active_learning_scores(\n",
        "            multiannotator_labels, pred_probs, pred_probs_unlabeled\n",
        "        )\n",
        "\n",
        "        # Get the indices of examples to collect more labels for.\n",
        "        chosen_examples_labeled, chosen_examples_unlabeled = get_idx_to_label(\n",
        "            X_labeled_full,\n",
        "            X_unlabeled,\n",
        "            extra_annotations,\n",
        "            batch_size_to_label,\n",
        "            active_learning_scores,\n",
        "            active_learning_scores_unlabeled,\n",
        "        )\n",
        "\n",
        "    # We don't need to run cross-validation, just get random examples to collect annotations for.\n",
        "    if selection_method == 'random':\n",
        "        chosen_examples_labeled, chosen_examples_unlabeled = get_idx_to_label_random(\n",
        "        X_labeled_full,\n",
        "        X_unlabeled,\n",
        "        extra_annotations,\n",
        "        batch_size_to_label\n",
        "        )\n",
        "\n",
        "    unlabeled_example_ids = np.array([])\n",
        "    # Check to see if we still have unlabeled examples left.\n",
        "    if X_unlabeled is not None:\n",
        "        # Get unlabeled text examples we want to collect annotations for.\n",
        "        new_text = X_unlabeled.iloc[chosen_examples_unlabeled]\n",
        "        unlabeled_example_ids = new_text.index.values\n",
        "        num_ex, num_annot = len(new_text), multiannotator_labels.shape[1]\n",
        "        empty_annot = pd.DataFrame(data = np.full((num_ex, num_annot), np.NaN), columns = multiannotator_labels.columns, index=unlabeled_example_ids)\n",
        "        new_unlabeled_df = pd.concat([new_text, empty_annot], axis=1)\n",
        "\n",
        "        # Combine unlabeled text examples with existing, labeled examples.\n",
        "        X_labeled_full = pd.concat([X_labeled_full, new_unlabeled_df], axis=0)\n",
        "\n",
        "        # Remove examples from X_unlabeled and check if empty.\n",
        "        # Once it is empty we set it to None to handle appropriately elsewhere.\n",
        "        X_unlabeled = X_unlabeled.drop(new_text.index)\n",
        "        if X_unlabeled.empty:\n",
        "            X_unlabeled = None\n",
        "\n",
        "    if selection_method == 'active_learning':\n",
        "        # Update pred_prob arrays with newly added examples if necessary.\n",
        "        if pred_probs_unlabeled is not None and len(chosen_examples_unlabeled) != 0:\n",
        "            pred_probs_new = pred_probs_unlabeled[chosen_examples_unlabeled, :]\n",
        "            pred_probs_labeled = np.concatenate((pred_probs, pred_probs_new))\n",
        "            pred_probs_unlabeled = np.delete(\n",
        "                pred_probs_unlabeled, chosen_examples_unlabeled, axis=0\n",
        "            )\n",
        "        # Otherwise we have nothing to modify.\n",
        "        else:\n",
        "            pred_probs_labeled = pred_probs\n",
        "\n",
        "    # Get combined list of text ID's to relabel.\n",
        "    labeled_example_ids = X_labeled_full.iloc[chosen_examples_labeled].index.values\n",
        "    combined_example_ids = np.concatenate([labeled_example_ids, unlabeled_example_ids])\n",
        "\n",
        "    # Now we collect annotations for the selected examples.\n",
        "    for example_id in combined_example_ids:\n",
        "        # Choose which annotator to collect annotation from.\n",
        "        chosen_annotator = get_annotator(example_id)\n",
        "        # Collect new annotation.\n",
        "        new_annotation = get_annotation(example_id, chosen_annotator)\n",
        "        # New annotator has been selected.\n",
        "        if chosen_annotator not in X_labeled_full.columns.values:\n",
        "            empty_col = np.full((len(X_labeled_full),), np.nan)\n",
        "            X_labeled_full[chosen_annotator] = empty_col\n",
        "\n",
        "        # Add selected annotation to the training set.\n",
        "        X_labeled_full.at[example_id, chosen_annotator] = new_annotation"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c0e624ce",
      "metadata": {
        "id": "c0e624ce"
      },
      "source": [
        "## Results\n",
        "\n",
        "After running 25 rounds of active learning (labeling batches of data and retraining the Transformer model), collecting 25 annotations in each round. I repeated all of this, the next time using random selection to choose which examples to annotate in each round — as a baseline comparison. Before additional data are annotated, both approaches start with the same initial training set of 100 examples (hence achieving roughly the same Transformer accuracy in the first round).  Because of inherent stochasticity in training Transformers, I ran this entire process five times (for each data labeling strategy) and report the standard deviation (shaded region) and mean (solid line) of test accuracies across the five replicate runs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1f03aac5",
      "metadata": {
        "id": "1f03aac5"
      },
      "outputs": [],
      "source": [
        "# Get numpy array of results.\n",
        "!wget -nc -O 'random_acc.npy' 'https://huggingface.co/datasets/Cleanlab/stanford-politeness/resolve/main/activelearn_acc.npy'\n",
        "!wget -nc -O 'activelearn_acc.npy' 'https://huggingface.co/datasets/Cleanlab/stanford-politeness/resolve/main/random_acc.npy'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9fddd368",
      "metadata": {
        "id": "9fddd368"
      },
      "outputs": [],
      "source": [
        "# Helper method to compute std dev across 2D array of accuracies.\n",
        "def compute_std_dev(accuracy):\n",
        "    def compute_std_dev_ind(accs):\n",
        "        mean = np.mean(accs)\n",
        "        std_dev = np.std(accs)\n",
        "        return np.array([mean - std_dev, mean + std_dev])\n",
        "\n",
        "    std_dev = np.apply_along_axis(compute_std_dev_ind, 0, accuracy)\n",
        "    return std_dev\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b1cfa980",
      "metadata": {
        "id": "b1cfa980",
        "outputId": "a8b836f2-2d6f-42c1-e038-54c86ea7e26d"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAD4VUlEQVR4nOydd3wUZf7H39t3s5tKeiGBEHqvIl1QlCLYEBtgOxt6FuyeivdTsKGeivUUT/AEFRBPBbHQUTqC9AAB0nuyvczz+2PIwpIEUkmAeb9e+0p29pmZZ2Z3Zz77rSohhEBBQUFBQUFB4QJC3dQTUFBQUFBQUFA42ygCSEFBQUFBQeGCQxFACgoKCgoKChccigBSUFBQUFBQuOBQBJCCgoKCgoLCBYcigBQUFBQUFBQuOBQBpKCgoKCgoHDBoQggBQUFBQUFhQsORQApKCgoKCgoXHAoAugCZMqUKaSkpDT1NGrMlClTsFgsTT2Nc4rDhw+jUqmYM2dOU09F4SRWrFiBSqVixYoVTT2VBkGlUvH888+f9f0+//zzqFSqs75fhfMLRQA1Q2bPno1KpaJfv3513kZWVhbPP/8827Zta7iJ1YKUlBTGjBnTJPs+26SkpKBSqfwPs9lM3759+c9//tPUU2uW+Hw+4uPjUalU/Pjjj009nUqsW7eO559/npKSkjpvY/bs2c1OfEqSxH/+8x/69etHREQEwcHBtG3blkmTJvH777839fQqYbfbef75588bsVgTTr6OnPyYOXPmGdedM2dOwDparZaEhASmTJlCZmbmWZj9uYe2qSegUJl58+aRkpLChg0bOHDgAG3atKn1NrKyspg+fTopKSl079494LWPPvoISZIaaLYKAN27d+eRRx4BIDs7m48//pjJkyfjcrm48847m3h2zYtff/2V7OxsUlJSmDdvHldccUVTTymAdevWMX36dKZMmUJYWFidtjF79mwiIyOZMmVKwPLBgwfjcDjQ6/X1n2gteeCBB3j33XcZN24cN910E1qtlr179/Ljjz/SunVrLrroorM+p9Nht9uZPn06AEOHDg147ZlnnuGJJ55oglk1PpdeeimTJk0KWNajR48ar//CCy/QqlUrnE4nv//+O3PmzGHNmjXs3LkTo9HY0NM9p1EEUDPj0KFDrFu3joULF3LXXXcxb948nnvuuQbdh06na9DtKUBCQgI333yz//mUKVNo3bo1b7zxhiKATmHu3Ln07NmTyZMn89RTT2Gz2TCbzU09rbOCWq1ukptQbm4us2fP5s477+TDDz8MeO3NN98kPz//rM+pPmi1WrTa8/P21bZt24BrSW254oor6N27NwB33HEHkZGRvPzyyyxZsoQJEyY01DTPCxQXWDNj3rx5hIeHM3r0aK699lrmzZtX5biSkhIeeughUlJSMBgMJCYmMmnSJAoKClixYgV9+vQB4NZbb/WbRCtM8ifHAHk8HiIiIrj11lsr7aOsrAyj0ci0adP8y1wuF8899xxt2rTBYDCQlJTEY489hsvlqvWxrl69muuuu46WLVv6t/XQQw/hcDiqHH/w4EFGjhyJ2WwmPj6eF154ASHEafcxZswYWrduXeVr/fv3918oAJYvX87AgQMJCwvDYrHQrl07nnrqqVofF0BUVBTt27cnPT09YHlNj7ki7ikzM5Px48djsViIiopi2rRp+Hy+gLElJSVMmTKF0NBQwsLCmDx5crXum19//ZVBgwZhNpsJCwtj3Lhx7N69O2BMRXzFvn37uPnmmwkNDSUqKop//OMfCCE4evQo48aNIyQkhNjYWF5//fUanxeHw8GiRYuYOHEiEyZMwOFw8O2331YaV9Pjr4h1eu211/jwww9JTU3FYDDQp08fNm7cWOvjf/7553n00UcBaNWqlf+7c/jwYQA+/fRTLrnkEqKjozEYDHTs2JH33nsvYB8pKSn89ddfrFy50r9+hQWjuhigr776il69emEymYiMjOTmm2+u5LaozWfiVA4dOoQQggEDBlR6TaVSER0dHbCspKSEBx98kKSkJAwGA23atOHll1+ukeU4MzOT2267jZiYGAwGA506deKTTz6pNM7pdPL888/Ttm1bjEYjcXFxXH311aSnp3P48GGioqIAmD59uv88VsQbVRUD5PV6+ec//+n/DKSkpPDUU09VujZVuOfXrFlD3759MRqNtG7dulm5rB0OB06ns0G2NWjQIICAa9HQoUMrWdWgcnxobb5fOTk53HrrrSQmJmIwGIiLi2PcuHH+706zRCg0K9q3by9uv/12IYQQq1atEoDYsGFDwJjy8nLRuXNnodFoxJ133inee+898c9//lP06dNHbN26VeTk5IgXXnhBAOJvf/ub+Pzzz8Xnn38u0tPThRBCTJ48WSQnJ/u3d9ttt4mwsDDhcrkC9vPZZ58JQGzcuFEIIYTP5xOXXXaZCAoKEg8++KD44IMPxNSpU4VWqxXjxo0LWDc5OVmMHj36tMd6//33i1GjRomXXnpJfPDBB+L2228XGo1GXHvttQHjJk+eLIxGo0hLSxO33HKLeOedd8SYMWMEIP7xj3+cdh//+c9/qjyHhw8fFoB49dVXhRBC7Ny5U+j1etG7d2/x1ltviffff19MmzZNDB48+LTbr+5YPR6PiI2NFTExMfU65k6dOonbbrtNvPfee+Kaa64RgJg9e7Z/nCRJYvDgwUKtVot7771XvP322+KSSy4RXbt2FYD49NNP/WOXL18utFqtaNu2rXjllVfE9OnTRWRkpAgPDxeHDh3yj3vuuecEILp37y5uuOEGMXv2bDF69GgBiFmzZol27dqJe+65R8yePVsMGDBAAGLlypVnPE9CCPHll18KlUoljhw5IoQQ4pJLLhGjRo2qNK6mx3/o0CEBiB49eog2bdqIl19+WbzyyisiMjJSJCYmCrfbXavj3759u7jhhhsEIN544w3/d8dqtQohhOjTp4+YMmWKeOONN8Tbb78tLrvsMgGId955x7+fRYsWicTERNG+fXv/+j/99JMQQojffvtNAOK3337zj//0008FIPr06SPeeOMN8cQTTwiTySRSUlJEcXFxrc9JVWRlZQlAjB49WthsttOOtdlsomvXrqJFixbiqaeeEu+//76YNGmSUKlU4u9//3vAWEA899xz/uc5OTkiMTFRJCUliRdeeEG899574sorr/Sfzwq8Xq8YPny4AMTEiRPFO++8I2bMmCEuueQSsXjxYmG1WsV7770nAHHVVVf5z+P27duFECc+oyczefJkAYhrr71WvPvuu2LSpEkCEOPHjw8Yl5ycLNq1aydiYmLEU089Jd555x3Rs2dPoVKpxM6dO097bhobQJjNZqFSqQQgOnToIObNm1ejdSs+RxXX6wreeecdAYj33nvPv2zIkCFiyJAhlbZx6r2hNt+viy++WISGhopnnnlGfPzxx+Kll14Sw4YNq/G1oSlQBFAzYtOmTQIQy5cvF0LIN7fExMRKF51nn31WAGLhwoWVtiFJkhBCiI0bN1a6AVZw6od82bJlAhDfffddwLhRo0aJ1q1b+59//vnnQq1Wi9WrVweMe//99wUg1q5d619WEwFkt9srLZsxY4ZQqVQiIyMjYL6AuP/++wOOc/To0UKv14v8/Pxq91FaWioMBoN45JFHApa/8sorAft54403BHDabVVHcnKyuOyyy0R+fr7Iz88XO3bsELfccosAxH333VevY37hhRcCxvbo0UP06tXL/3zx4sUCEK+88op/mdfrFYMGDar0/nfv3l1ER0eLwsJC/7Lt27cLtVotJk2a5F9WcXP529/+FrDNxMREoVKpxMyZM/3Li4uLhclkEpMnT67BmRJizJgxYsCAAf7nH374odBqtSIvLy9gXE2Pv+IC3aJFC1FUVORf/u2331b6TNf0+F999VUBBIjCCqp6/0aOHBnwPRFCiE6dOlV5gzlVALndbhEdHS06d+4sHA6Hf9z//vc/AYhnn3221uekOioEQXh4uLjqqqvEa6+9Jnbv3l1p3D//+U9hNpvFvn37ApY/8cQTQqPR+MWrEJUF0O233y7i4uJEQUFBwLoTJ04UoaGh/vP3ySef+AX1qVRcw/Lz8yttv4JTBdC2bdsEIO64446AcdOmTROA+PXXX/3LkpOTBSBWrVrlX5aXl1fldeJsc/HFF4s333xTfPvtt+K9994TnTt3rpHAFeKEAPr5559Ffn6+OHr0qPj6669FVFSUMBgM4ujRo/6xtRVAZ/p+FRcXB/ygPFdQXGDNiHnz5hETE8OwYcMA2TR9/fXX8+WXXwaYuL/55hu6devGVVddVWkbdUkNveSSS4iMjGT+/Pn+ZcXFxSxfvpzrr7/ev+yrr76iQ4cOtG/fnoKCAv/jkksuAeC3336r1X5NJpP/f5vNRkFBARdffDFCCLZu3Vpp/NSpU/3/q1Qqpk6ditvt5ueff652HyEhIVxxxRUsWLAgwF02f/58LrroIlq2bAngD3b99ttv6xQg/tNPPxEVFUVUVBRdunTh888/59Zbb+XVV1+t1zHffffdAc8HDRrEwYMH/c9/+OEHtFot99xzj3+ZRqPh/vvvD1gvOzubbdu2MWXKFCIiIvzLu3btyqWXXsoPP/xQad933HFHwDZ79+6NEILbb7/dvzwsLIx27doFzKk6CgsLWbZsGTfccIN/2TXXXINKpWLBggVVrnOm46/g+uuvJzw8PGAc4B9bl+OvipPfv9LSUgoKChgyZAgHDx6ktLS0Rts4mU2bNpGXl8e9994bEBs0evRo2rdvz/fff19pnZqek1P59NNPeeedd2jVqhWLFi1i2rRpdOjQgeHDhwe427766isGDRpEeHh4wPd8xIgR+Hw+Vq1aVeX2hRB88803jB07FiFEwLojR46ktLSULVu2API1LDIystLnFOp2Dat4/x5++OGA5RWJCaeex44dO/o/IyC7rGv6OW5M1q5dy9///neuvPJK7r77bjZv3kznzp156qmnqg0NOJURI0YQFRVFUlIS1157LWazmSVLlpCYmFjneZ3p+2UymdDr9axYsYLi4uI67+dsowigZoLP5+PLL79k2LBhHDp0iAMHDnDgwAH69etHbm4uv/zyi39seno6nTt3brB9a7VarrnmGr799lu/v3zhwoV4PJ4AAbR//37++usv/42+4tG2bVsA8vLyarXfI0eO+G9IFfEMQ4YMAah0M1Gr1ZVieSr2eyYf8/XXX8/Ro0dZv349IJ+/zZs3Bxzb9ddfz4ABA7jjjjuIiYlh4sSJLFiwoMZiqF+/fixfvpylS5fy2muvERYWRnFxcaVsn9ocs9Fo9MdBVBAeHh5wgcnIyCAuLq5SnaR27doFPM/IyKhyOUCHDh0oKCjAZrMFLK8QhxWEhoZiNBqJjIystLwmF7358+fj8Xjo0aOH//NdVFREv379qox1q8nxVzfXiot1xdi6HH9VrF27lhEjRvhjiKKiovxxYnURQKebV/v27f2vV1Cbc3IqarWa++67j82bN1NQUMC3337LFVdcwa+//srEiRP94/bv38/SpUsrfc9HjBgBVP89z8/Pp6SkhA8//LDSuhUxhhXrpqen065duwYLZM7IyECtVlfKmI2NjSUsLKzSeTz18wI1O4/5+fnk5OT4H1arFSBgWU5ODg6HA5/PV2m52+2u1XHp9XqmTp1KSUkJmzdvrtE67777LsuXL+frr79m1KhRFBQUYDAYarXfUznT98tgMPDyyy/z448/EhMTw+DBg3nllVfIycmp134bm/MzjP4cpCI1+Msvv+TLL7+s9Pq8efO47LLLGm3/EydO5IMPPuDHH39k/PjxLFiwgPbt29OtWzf/GEmS6NKlC7NmzapyG0lJSTXen8/n49JLL6WoqIjHH3+c9u3bYzabyczMZMqUKQ2apj927FiCgoJYsGABF198MQsWLECtVnPdddf5x5hMJlatWsVvv/3G999/z9KlS5k/fz6XXHIJP/30ExqN5rT7iIyM9N8gRo4cSfv27RkzZgxvvfWW/1dpbY/5TPtsbKraf3VzOtm6Vh0VIqeqQFyQf02eLHJrc/z1mVdNSU9PZ/jw4bRv355Zs2aRlJSEXq/nhx9+4I033jgrpSUa6jPRokULrrzySq688kqGDh3KypUrycjIIDk5GUmSuPTSS3nssceqXLfih8epVBz/zTffzOTJk6sc07Vr1waZf3XU1HpU189Lnz59AsTUc889x/PPP09cXFzAuE8//ZShQ4fSqlWrgOW//fZblcHHp6PiulpUVFSj8X379vUnd4wfP56BAwdy4403snfvXv8PJZVKVeWxVhdMX5Pz9eCDDzJ27FgWL17MsmXL+Mc//sGMGTP49ddfa5XGfzZRBFAzYd68eURHR/Puu+9Wem3hwoUsWrSI999/H5PJRGpqKjt37jzt9mprRh48eDBxcXHMnz+fgQMH8uuvv/L0008HjElNTWX79u0MHz683lVYd+zYwb59+/jss88Cal4sX768yvGSJHHw4MGAi+++ffsAzljV2mw2M2bMGL766itmzZrF/PnzGTRoEPHx8QHj1Go1w4cPZ/jw4cyaNYuXXnqJp59+mt9++80vbmrK6NGjGTJkCC+99BJ33XUXZrO51sdcE5KTk/nll1+wWq0BVqC9e/dWGlfVcoA9e/YQGRnZqKnoFeUdpk6d6rd4VSBJErfccgtffPEFzzzzTKPsvzbHX91n+7vvvsPlcrFkyZKAX8RVuX5r+v04eV4VruQK9u7d63+9MenduzcrV64kOzub5ORkUlNTsVqttf7MR0VFERwcjM/nO+O6qamp/PHHH3g8nmrLctTmGlMh3Pbv30+HDh38y3NzcykpKWmw8zhv3rwAV1SFYD/1O9ypUyfCw8MrLT/5B2VNqXAznWr5qwkajYYZM2YwbNgw3nnnHX/tpPDw8CrdfadaympLamoqjzzyCI888gj79++ne/fuvP7668ydO7de220sFBdYM8DhcLBw4ULGjBnDtddeW+kxdepUysvLWbJkCSDHTWzfvp1FixZV2laFIq+4mNe0mq1arebaa6/lu+++4/PPP8fr9Qa4iAAmTJhAZmYmH330UZXHUBMXQgUVvyhO/gUhhOCtt96qdp133nknYOw777yDTqdj+PDhZ9zf9ddfT1ZWFh9//DHbt2+vdGxV/bqqKCBZlxR/gMcff5zCwkL/+arLMZ+JUaNG4fV6A1KxfT4fb7/9dsC4uLg4unfvzmeffRbwmdi5cyc//fQTo0aNqvMcakKF9eexxx6r9PmeMGECQ4YMqbbkQ0NQm+Ov7rtT1ftXWlrKp59+Wml/ZrO5Rt+93r17Ex0dzfvvvx/wOfvxxx/ZvXs3o0ePrsnhnZGcnBx27dpVabnb7eaXX34JcB9NmDCB9evXs2zZskrjS0pK8Hq9Ve5Do9FwzTXX8M0331T5A+3kWkPXXHMNBQUFAd/pCirOb1BQkH+fZ6Li/XvzzTcDlldYqxvqPA4YMIARI0b4HxUC6ORlI0aMIC4uDqPRWGn5yXE0p1JVLaby8nLefPNNIiMj6dWrV53mPHToUPr27cubb77pT61PTU1lz549Afvcvn07a9eurdM+7HZ7pbT91NRUgoOD63z9PBsoFqBmwJIlSygvL+fKK6+s8vWLLrqIqKgo5s2bx/XXX8+jjz7K119/zXXXXcdtt91Gr169KCoqYsmSJbz//vt069aN1NRUwsLCeP/99wkODsZsNtOvX79KJtmTuf7663n77bd57rnn6NKlS8AvKYBbbrmFBQsWcPfdd/Pbb78xYMAAfD4fe/bsYcGCBSxbtiygrs6BAwf4v//7v0r76dGjB5dddhmpqalMmzaNzMxMQkJC+Oabb6r1wRuNRpYuXcrkyZPp168fP/74I99//z1PPfVUjX4ZjRo1iuDgYKZNm+a/UJ/MCy+8wKpVqxg9ejTJycnk5eUxe/ZsEhMTGThw4Bm3XxVXXHEFnTt3ZtasWdx33320b9++VsdcE8aOHcuAAQN44oknOHz4MB07dmThwoVVxqO8+uqrXHHFFfTv35/bb78dh8PB22+/TWhoaKP3c5o3bx7du3ev1k165ZVXcv/997NlyxZ69uzZKHOo6fFX3GiefvppJk6ciE6nY+zYsVx22WXo9XrGjh3LXXfdhdVq5aOPPiI6Oprs7OyAffXq1Yv33nuP//u//6NNmzZER0dXsvCAXJT05Zdf5tZbb2XIkCHccMMN5Obm8tZbb5GSksJDDz3UIMd+7Ngx+vbtyyWXXMLw4cOJjY0lLy+P//73v2zfvp0HH3zQH9v16KOPsmTJEsaMGcOUKVPo1asXNpuNHTt28PXXX3P48OFKcWAVzJw5k99++41+/fpx55130rFjR4qKitiyZQs///yz/4fGpEmT+M9//sPDDz/Mhg0bGDRoEDabjZ9//pl7772XcePGYTKZ6NixI/Pnz6dt27ZERETQuXPnKuMfu3XrxuTJk/nwww8pKSlhyJAhbNiwgc8++4zx48f7E0uaM++++y6LFy9m7NixtGzZkuzsbD755BOOHDnC559/Xq/q4Y8++ijXXXcdc+bM4e677+a2225j1qxZjBw5kttvv528vDzef/99OnXqRFlZWa23v2/fPoYPH86ECRPo2LEjWq2WRYsWkZubGxBf1uw4y1lnClUwduxYYTQaT1ufY8qUKUKn0/nTSwsLC8XUqVNFQkKC0Ov1IjExUUyePDkg/fTbb78VHTt2FFqtNiAl+tRUxwokSRJJSUkCEP/3f/9X5Tzcbrd4+eWXRadOnYTBYBDh4eGiV69eYvr06aK0tNQ/riLVtKpHRZ2jXbt2iREjRgiLxSIiIyPFnXfeKbZv314pfXvy5MnCbDaL9PR0fx2imJgY8dxzzwmfz1fT0yxuuukmAYgRI0ZUeu2XX34R48aNE/Hx8UKv14v4+Hhxww03VEoFrorTpfzPmTMn4Hhqe8ynUlX9k8LCQnHLLbeIkJAQERoaKm655RaxdevWKssg/Pzzz2LAgAHCZDKJkJAQMXbsWLFr164q93FqSYDq5jRkyBDRqVOn6k6P2Lx58xlrNlXUZXrooYdqdfwVabpVpd9SRQp1TY5fCDkVPCEhQajV6oCU+CVLloiuXbsKo9EoUlJSxMsvv+xP6T45bT4nJ0eMHj1aBAcHC8CfclxVHSAhhJg/f77o0aOHMBgMIiIiQtx0003i2LFjAWNq85k4lbKyMvHWW2+JkSNHisTERKHT6URwcLDo37+/+Oijj/yp5xWUl5eLJ598UrRp00bo9XoRGRkpLr74YvHaa68F1H6p6hzn5uaK++67TyQlJQmdTidiY2PF8OHDxYcffhgwzm63i6efflq0atXKP+7aa6/11ysTQoh169aJXr16Cb1eH7Cvqo7Z4/GI6dOn+7eXlJQknnzySeF0OgPGVfd9rS41/Gzx008/iUsvvVTExsYKnU4nwsLCxGWXXSZ++eWXGq1fXR0gIeQabqmpqSI1NVV4vV4hhBBz584VrVu3Fnq9XnTv3l0sW7as2jT4M32/CgoKxH333Sfat28vzGazCA0NFf369RMLFiyo/Yk4i6iEaMAoQQUFBQUFBQWFcwAlBkhBQUFBQUHhgkMRQAoKCgoKCgoXHIoAUlBQUFBQULjgUASQgoKCgoKCwgWHIoAUFBQUFBQULjgUAaSgoKCgoKBwwaEUQqwCSZLIysoiODi43i0fFBQUFBQUFM4OQgjKy8uJj49HrT69jUcRQFWQlZVVq8aeCgoKCgoKCs2Ho0ePkpiYeNoxigCqguDgYEA+gSEhIU08GwUFBQUFBYWaUFZWRlJSkv8+fjoUAVQFFW6vkJAQRQApKCgoKCicY9QkfEUJglZQUFBQUFC44FAEkIKCgoKCgsIFR5MKoFWrVjF27Fji4+NRqVQsXrz4jOusWLGCnj17YjAYaNOmDXPmzKk05t133yUlJQWj0Ui/fv3YsGFDw09eQUFBQUFB4ZylSWOAbDYb3bp147bbbuPqq68+4/hDhw4xevRo7r77bubNm8cvv/zCHXfcQVxcHCNHjgRg/vz5PPzww7z//vv069ePN998k5EjR7J3716io6MbdP4+nw+Px9Og21RQ0Ov1Z0zfVFBQUFCoHyohhGjqSYAcsLRo0SLGjx9f7ZjHH3+c77//np07d/qXTZw4kZKSEpYuXQpAv3796NOnD++88w4g1/RJSkri/vvv54knnqjRXMrKyggNDaW0tLTKIGghBDk5OZSUlNT8ABUUaoharaZVq1bo9fqmnoqCgoLCOcWZ7t8nc05lga1fv54RI0YELBs5ciQPPvggAG63m82bN/Pkk0/6X1er1YwYMYL169dXu12Xy4XL5fI/LysrO+08KsRPdHQ0QUFBSrFEhQajoghndnY2LVu2VD5bCgoKCo3EOSWAcnJyiImJCVgWExNDWVkZDoeD4uJifD5flWP27NlT7XZnzJjB9OnTazQHn8/nFz8tWrSo/UEoKJyBqKgosrKy8Hq96HS6pp6OgoKCwnmJEmgAPPnkk5SWlvofR48erXZsRcxPUFDQ2ZqewgVGhevL5/M18UwUFBQUzl/OKQtQbGwsubm5Actyc3MJCQnBZDKh0WjQaDRVjomNja12uwaDAYPBUKu5KK4JhcZC+WwpKCgoND7nlAWof//+/PLLLwHLli9fTv/+/QH5l3OvXr0CxkiSxC+//OIfo6CgoKCgoKDQpALIarWybds2tm3bBshp7tu2bePIkSOA7JqaNGmSf/zdd9/NwYMHeeyxx9izZw+zZ89mwYIFPPTQQ/4xDz/8MB999BGfffYZu3fv5p577sFms3Hrrbee1WO7UKhp/abmvo+aMnToUH/QvYKCgoLCuUuTusA2bdrEsGHD/M8ffvhhACZPnsycOXPIzs72iyGAVq1a8f333/PQQw/x1ltvkZiYyMcff+yvAQRw/fXXk5+fz7PPPktOTg7du3dn6dKllQKjFWrH888/z+LFi/1itYLs7GzCw8ObZlJNwMKFC5XAZAUFBYXzgCYVQEOHDuV0ZYiqqvI8dOhQtm7detrtTp06lalTp9Z3ego14HSxVecSHo+nRsImIiLiLMxGQUFBQaGxOadigBTqztKlSxk4cCBhYWG0aNGCMWPGkJ6eHjDm2LFj3HDDDURERGA2m+nduzd//PEHc+bMYfr06Wzfvh2VSoVKpfKL05PdUxdffDGPP/54wDbz8/PR6XSsWrUKkGsuTZs2jYSEBMxmM/369WPFihW1OpajR48yYcIEwsLCiIiIYNy4cRw+fNj/+saNG7n00kuJjIwkNDSUIUOGsGXLloBtqFQq3nvvPa688krMZjMvvvgizz//PN27d+fzzz8nJSWF0NBQJk6cSHl5uX+9U11gKSkpvPTSS9x2220EBwfTsmVLPvzww4B9rVu3ju7du2M0GunduzeLFy9GpVJVsqYpKCgoNDQur4siRxF2j/20BocLkXMqC6y5IoTA7rGf9f0G6WpehNFms/Hwww/TtWtXrFYrzz77LFdddRXbtm1DrVZjtVoZMmQICQkJLFmyhNjYWLZs2YIkSVx//fXs3LmTpUuX8vPPPwMQGhpaaR833XQTr7zyCjNnzvTPa/78+cTHxzNo0CBAts7t2rWLL7/8kvj4eBYtWsTll1/Ojh07SEtLO+NxeDweRo4cSf/+/Vm9ejVarZb/+7//4/LLL+fPP/9Er9dTXl7O5MmTefvttxFC8PrrrzNq1Cj2799PcHCwf1vPP/88M2fO5M0330Sr1fLJJ5+Qnp7O4sWL+d///kdxcTETJkxg5syZvPjii9XO6fXXX+ef//wnTz31FF9//TX33HMPQ4YMoV27dpSVlTF27FhGjRrFF198QUZGhhJDpKCg0GhY3VaKHEUUOYrILMskz5aHw+tAr9ETpA0i3BROC1MLLAYLZp0Zs96MRW/BoDFccBmoigBqAOweO5YZlrO+X+uTVsx6c43GXnPNNQHPP/nkE6Kioti1axedO3fmiy++ID8/n40bN/rdPG3atPGPt1gsaLXa07q8JkyYwIMPPsiaNWv8gueLL77ghhtuQKVSceTIET799FOOHDlCfHw8ANOmTWPp0qV8+umnvPTSS2c8jvnz5yNJEh9//LH/y/rpp58SFhbGihUruOyyy7jkkksC1vnwww8JCwtj5cqVjBkzxr/8xhtvrBQcL0kSc+bM8QulW265hV9++eW0AmjUqFHce++9gNyu5Y033uC3336jXbt2fPHFF6hUKj766COMRiMdO3YkMzOTO++884zHqqCgoHA6JCFR4iyhyFFEgb2AzLJMihxFWN1WfMKHQWMgWB9MC1ML3D43No+NIkcRu3y7ANkSbtAYMGqNBOmDaGFsQURQhF8YVfw1ao1NfKSNgyKALhD279/Ps88+yx9//EFBQQGSJAFw5MgROnfuzLZt2+jRo0e9YlyioqK47LLLmDdvHoMGDeLQoUOsX7+eDz74AIAdO3bg8/lo27ZtwHoul6vGVbW3b9/OgQMHAiw5AE6n0+/Sy83N5ZlnnmHFihXk5eXh8/mw2+0BAfUAvXv3rrT9lJSUgG3HxcWRl5d32jl17drV/79KpSI2Nta/zt69e+natStG44kLSN++fWt0rAoKCgon4/a5/dadfFs+x8qOUeYqw+axoUJFkC6IYH0wUUFRaNSagHWDdJWL90pCwuV14fQ5sbqsFNgK8OR7EEKgQoVRZ8SoNWLWmYkwRRBhisCoNQZYilSc9P8Zlp+6LNocTYjh9P26GhNFADUAQbogrE9am2S/NWXs2LEkJyfz0UcfER8fjyRJdO7cGbfbDYDJZGqQOd1000088MADvP3223zxxRd06dKFLl26AHLZA41Gw+bNm9FoAr+cFkvNLGhWq5VevXoxb968Sq9FRUUBchZhYWEhb731FsnJyRgMBvr37+8/1grM5srWs1MDoVUqlV8sVkdd1lFQUFA4Eza3zS94ssqzyLHmUO4ux+PzoFFrMOvMhBvDSQhOqJP7Sq1SY9KZMOkqX/8lIeH0OnF5XZQ6S8m15uKW3Cf2c3I40cm7PtPy48t8ko9LWl1C7/jKP0TPFooAagBUKlWNXVFNQWFhIXv37uWjjz7yu6bWrFkTMKZr1658/PHHFBUVVWkF0uv1NWrNMG7cOP72t7+xdOlSvvjii4A6Tj169MDn85GXl+efR23p2bMn8+fPJzo6utpOv2vXrmX27NmMGjUKkIOmCwoK6rS/+tKuXTvmzp2Ly+XyVxvfuHFjk8xFQUGh+eLxeShxllDsLPbH71S4syQhodfoCdYHE2+Jx6CtXeeCuqBWqQnSBdXqh3Zt2Fe4r1G2WxuULLALgPDwcFq0aMGHH37IgQMH+PXXX/01lyq44YYbiI2NZfz48axdu5aDBw/yzTffsH79ekB2DVUUqiwoKMDlclW5L7PZzPjx4/nHP/7B7t27ueGGG/yvtW3blptuuolJkyaxcOFCDh06xIYNG5gxYwbff/99jY7lpptuIjIyknHjxrF69WoOHTrEihUreOCBBzh27BgAaWlpfP755+zevZs//viDm266qcEsXLXlxhtvRJIk/va3v7F7926WLVvGa6+9BigtLxQULlQqYncOlxxma/ZWftj/A/P+nMeCvxbw3d7vWHtkLXm2PIxaIylhKbRt0ZaUsBRaBLU4K+LnQkERQBcAarWaL7/8ks2bN9O5c2ceeughXn311YAxer2en376iejoaEaNGkWXLl2YOXOm31V1zTXXcPnllzNs2DCioqL473//W+3+brrpJrZv386gQYNo2bJlwGuffvopkyZN4pFHHqFdu3aMHz+ejRs3VhpXHUFBQaxatYqWLVty9dVX06FDB26//XacTqffIvTvf/+b4uJievbsyS233MIDDzxAdHR0bU5ZgxESEsJ3333Htm3b6N69O08//TTPPvssQEBckIKCwvmL3WMnsyyTv/L+YsWhFXy540u+3PklC3cv5NdDv5JelI5AEBUURVpEGm1btCUxJJEwYxhateKoaSxUQikMUImysjJCQ0MpLS2t5GZxOp0cOnSIVq1aKTcwhToxb948br31VkpLS6u0TCmfMQWFc5eTXVmF9kKyyrModhRj9VjxSl60Kq0/9TxIF4Ra1Xh2iBJnCd/u/Ra7x05qeCqpEam0DGmJTtP01ez3Fe5jSMqQBo8BOt39+1QUaamg0Mj85z//oXXr1iQkJLB9+3Yef/xxJkyY0GRuOQUFhRO4vC5/2rhP8uETPiQh+f/3Scefn/S6T5IfbsmNx+fBK3nxSl6/+LF6rLi8cphAkC4Ii95ChCnirAkPr+Tl+/3f89+d/61Uo06r1pISlkKb8DakRqTSJqJNsxFFZxtFACkoNDI5OTn+3nRxcXFcd911p60rpKCg0Lh4fB6yrdkcKT1CelG6P9D45IdKpZIrJ6uQM5gqQvaO+0xUqFCr1AEPlUqFUWskxhzTZLVztmRv4eOtH3OsTI6JbBPehi4xXUgvSie9OB2bx8aBogMcKDoAx5sBVBJF4W1oGXr+iyJFACkoNDKPPfYYjz32WFNPQ0HhgsYn+ciz5XG09Cj7ivZRYC/AJ/kIN4YTbY4+IWJQVaqhcy6QXZ7Nv7f9mw2ZGwAINYQyqdskhrca7nezCSHIseaQXpzOgaID/r/ViqLQFFIjUkkNly1FyaHJ6DQ6vJLXnyLv9Dpx+VwnnvucAa+d/LrT68Ttc+P0OilyFFHiLFHS4BUUFBQUFBoaIYRcIbk8k32F+8ix5uDxeQgxhJAUkoReo2/qKdYbh8fBV7u+YvHexXglLxqVhjFtxzCx08RK5VlUKhVxwXHEBccxsOVAQD5HubbcAEGUXixbxQ4UH+BA8QH/+hqVBpVKhVfyNsjcj5QdOfOgRkQRQAoKCgoK5xXFjmIyyzPZX7Sf7PJsbG4bwfpg4ixx501bByEEKzNWMmf7HIocRQD0iO3BHT3uICk0qcbbUalUxFpiibXEVhJFfkF03H1W7i4PKHSoVqkxao0YNUYMWrmlhkFr8LfXMGqNgf9XjNEYKHYUc0WbKxr0nNQWRQApKCgoKJzzlLvKySzPJL0onWNlxyh3l2PSmmhhakFSSM0FwbnAgaIDfLjlQ/YU7AEg1hzL7T1vp2983wapL3ayKBqQNACQRVGhoxAhhF/QaNXaOu9vX+E+2kS0OfPARkQRQAoKCgoK5yQOj4PM8kwOFR8ioySDUlcpOrWOFkEtiLPEnXfFRkucJcz9cy7LDy5HIAuR6zpex7h24xrdnadSqYgMimzUfZxtFAGkoKCgoNDscXqd2Nw2bB4bNreNPHse6UXpFDuK0ag1RBgjaBPRplHr6jQVFWntX+78EpvHBsCQ5CFM6TaFFkE1ayStUBlFACkoKCgoNDlCCFw+V4DIsXlsFNmLKHQWYnfbcfqcuDwuBAKVSkW4MZzU8NRzMmurpmzN2crHWz7maNlRAFLDU7mz5510jOpYo/V9ko8yVxkmnem8iX9qKBQBpNAoTJkyhZKSEhYvXtzUU1FQUGgmVCVyrG4rxY7iyiJHJUCAXqP3B9AG64MxBBvOSyvPqeRYc/j31n/zR+YfAIQYQpjUVU5rr6ng80pe0ovTCTOGUWYrw+l1olVpCTGEEGIIueD7iikCSEFBQaGOHC09Sq41ly4xXS74m8mZOFZ2jN+P/k6pq1QWOV7ZknMhi5wKrG4rWeVZZJdnk23N5ljZMdYfW49H8qBWqRmTNoaJnSdi0VtqvE2Pz0N6cTqtw1szJGUIkpAosBeQXZ7N0bKjZFuzcXld6DQ6gvXBF6QgUgTQBYzb7UavP/frYCgonG2cXifbcraxJXsL5a5ycm25DEkZUqsb1IWCEIK9hXtZnbEam8dGhClCFjnaC0vklLvKybZmy0LnpL/Z5dlyenkVdIvpxp0976RlaM2aRVfg8ro4WHKQ9pHtGZYyjGBDMADR5mg6RnXE4/NQ6Cj09yrLLMskqzwLt+RGr9YTYggh2BDc4IHVkpD8LUMaqpZQfVAE0AXE0KFD6dy5M1qtlrlz59KlSxfGjh3Lp59+ysGDB4mIiGDs2LG88sorWCzyhXzOnDk8+OCDzJ8/nwcffJCjR48ycOBAPv30U+Li4gDw+Xw8+uijfPLJJ2g0Gm6//XZO7bHrcrl49NFH+fLLLykrK6N379688cYb9OnTB4AVK1YwbNgwli5dyhNPPMGePXvo37+/v4v9ww8/TGZmJmPGjOHjjz8mKCjo7J48BYXjZJZlsv7Yeg4VHyLGHEN0UDS78nfh9DoZ1moYEaaIpp4iGSUZmHQmos3RTToPn+Rjc/Zmfj/6O0atkdTw1CadT2MihJAtOVbZknOywMm2Vi9yKogwRhAbHEu8JZ644DjaRrSla0zXWmeyOTwOMkoz6BzVmSEpQyoVQwTQaXT+NPdO0Z1w+9wU2gspdJwQRMfKjuHxedBrZUEUog+psjWGJCQ8Pg8eSX54fV7573Gh4xM+/1gVKrQaLTq1jsigSAyaprU4KQKoIRAC7PYzj2togoKgll+Ozz77jHvuuYe1a9cC8OOPP/Kvf/2LVq1acfDgQe69914ee+wxZs+e7V/Hbrfz2muv8fnnn6NWq7n55puZNm0a8+bNA+D1119nzpw5fPLJJ3To0IHXX3+dRYsWcckll/i38dhjj/HNN9/w2WefkZyczCuvvMLIkSM5cOAAEREnbhjPP/8877zzDkFBQUyYMIEJEyZgMBj44osvsFqtXHXVVbz99ts8/vjj9TlzCgq1xuV1+a0+Lp+L1PBU/w2hTUQbDhYf5Mf9PzK89XBiLbFNMkePz8OW7C1szNqIXqPnosSL6BTVqUmChJ1eJ+uOrGNLzhaig6IJN4Wf9Tk0BB6fh1JXKcWOYoqdxx8n/V/iKPH/7/a5T7utCFMEcZY44oPjA/7GWmIx6erfHNnmtnG07CjdYrsxOHlwjYOe9Rq9v0J05+jOuLwuCh2FchXtsky5b1rZEbw+L2q1OqBHmkqlQqfWodPo/H+DDcFY9BZ/x/uK4oin/m3qoGyVOPWnugJlZWWEhoZSWlpKSEhIwGtOp5NDhw7RqlUrjMbjb57NBpYmMH1brWCurO6rY+jQoZSVlbFly5Zqx3z99dfcfffdFBQUALIF6NZbb+XAgQOkpsq/3mbPns0LL7xATk4OAPHx8Tz00EM8+uijAHi9Xlq1akWvXr1YvHgxNpuN8PBw5syZw4033giAx+MhJSWFBx98kEcffdRvAfr5558ZPnw4ADNnzuTJJ58kPT2d1q1bA3D33Xdz+PBhli5dWsuTde5Q5WdMoUnJKs/i96O/k16cTrQ5ukorjyQkDpUcIsQQwrCUYbQKb3VW51jqLGXtkbXsyt9FtCXaf+PuFNWJixIvItQYelbnsjJjJXsL9tIytCVBuuZpsXV5Xewv2k+hvbBKcVPsKD6j5eZUIkwRfitOhcCJC45r9CrU5a5ysqxZ9IrrxYCWAxrUfeX0Oim0y4LI5rH5qzlXJWz0Gn2TujZPd/8+FcUCdIHRq1evgOc///wzM2bMYM+ePZSVleH1enE6ndjtdr+bKSgoyC9+AOLi4sjLywOgtLSU7Oxs+vXr539dq9XSu3dvvxssPT0dj8fDgAED/GN0Oh19+/Zl9+7dAfPp2rWr//+YmBiCgoL84qdi2YYNG+p7GhQUaoTb52Z7znY2ZW3C7XMHWH1ORa1S0zqsNUfLjrIsfRlDkofQPrL9WSnGd6T0CKszVpNVnkWrsFb+YNZgfTB/5v5Jvj2f/on9aR3eutHnk12ezYrDK8gszzzt+WoKhBAcLTvKluwtbMnewl/5f+GRPGdcT6PSEGYMI9wYTrgpnHBjuPz8+P/+v8bwJgkkLnGWkG/Pp19CP/on9Uerbthbu1FrJCEkgYTgePD5QHt+SIfz4yiamqAg2RrTFPutJeaTLEaHDx9mzJgx3HPPPbz44otERESwZs0abr/9dtxut18A6XSBFzCVSlUpxqehOHlfKpWqyn1LktQo+1ZQOJkcaw7rj60nvSidyKBIEkMSz7iOSqWiZWhLcqw5LE9fjt1jp0dcj0b7ReyTfPyZ+ye/H/sdj+ShbYu2Afsy6Uy0bdGWY2XH+H7/9/SK60Wv+F6NZonYX7iflRkrsbqttAlv0yzq81jdVv7M/VMWPTlbKLAXBLzewtSC+OD4SkImzCQLnghTBBa9pdkGbBc6Cil2FHNx0sX0ie/TeOfc54OCAigrA6NR9noYjWAwgLp5npszoQighkClqpUrqrmwefNmJEni9ddfR338A7xgwYJabSM0NJS4uDj++OMPBg8eDMgusM2bN9OzZ08AUlNT0ev1rF27luTkZEB2gW3cuJEHH3yw4Q5IQaEB8Pg8/Jn7J5uyNmH32Gkd1rrWVoxYSyyFjkJWZqzE4XXQL6Ffg1tCbG4b64+t58/cPwk3hpMYVLVAU6vUtAxtSamzlHVH15FjzWFAywENGqckCYmt2VtZf2w9WpW2SYOdJSGRXpzut/LsLdyLJE78aNKpdXSO7kzPuJ70jO1JYkhig1nFVG4PKiEQKhVCo5aFQSNa3PJt+Vg9VgYnD25UoY3HA3l5UFwMJpMc81paKh+fwSCLIZNJ/l+vb9RjbkgUAXQB06ZNGzweD2+//TZjx45l7dq1vP/++7Xezt///ndmzpxJWloa7du3Z9asWZSUlPhfN5vN3HPPPTz66KNERETQsmVLXnnlFex2O7fffnsDHpGCQv3Iteay/th6DhQeIDIokvjg+Dpvq4WpBXq1nt+P/o7D42BQ8qAGs7zkWHNYlbGKwyWHSQ5NrlGMTagxlCBdEBmlGRTuLWywAGmX18X6Y+vZkrWFiKAIWpjOfmuGYmcx27K3sSVnC1tztlLmKgt4PTEkkR6xPegZ15POUZ0b3k0lBFqbA31hCSqvD9QqhFqNUKlAo0FSqxFaNUKnQ6hVUPGaWo3QyP9XLK+peMguz8YtuRmWMozO0Z0bz7XpdEJOjuzlCAsLtPZIErhcUFgo/6/TySIoOPiEdagZu8ua78wUGp1u3boxa9YsXn75ZZ588kkGDx7MjBkzmDRpUq2288gjj5Cdnc3kyZNRq9XcdtttXHXVVZSWlvrHzJw5E0mSuOWWWygvL6d3794sW7aM8PBzMzNE4fzC4/OwM28nG7I2YHPbaBXeqkGCSIMNwSSpk9iWsw2n18nQlKH+mix1QRISewr2sPbIWmxuG20j2tZKwOg0OtpEtCHfls/y9OVkl2fXK0C6zFXGqoxV7M7fTWJI4lmrg+SVvOwp2ON3ax0sPhjwuklroltsN3rG9qRHbA9iLDGNNheV14eupAx9STmSVos3yIhKkkASqIRA5fGiERIqnwAhoUJ1PINKgEqNUHNCDKlUCK0GodEgdFq8ZhOSofLn8FjZMQBGtB5B+8j2jXZs2Gyy+HG5IDy8sjhTq2XLj+l4BpvHA263vI5KJVuDTCbZQ2I0ys+bkbtMyQKrglpngSkoNCDKZ+zskmfL449jf7CvcB/hxnCizFENvg+3z83B4oOkhKVwSatL6tTA0ul18kfmH2zN2opZb663C6uiXkxccBwXJV5EanhqrawIudZcVmasJKMkIyDwuiqEEKw+spp9hfvwSl65RkxFrZjjf70+r/+1gOVVPPf4PHIV6ZNIDU/1W3naR7Zv8EDgqlA7XegLS9HaHXiDjIjaWjuEOC6UjgsmSUIlCRACtU9C0mlwR4TitQTJAkkIjpQewaA1MKzVMNpEtGmcAwM51icnR7bsnCGbqkqEkIWT2w1eL2g0sgAKDg50lzX4tJUsMAUFhfOMclc5XsmLTqNDq5aLqdXHfeOVvOzM3cnGrI2Uu8tJCUtp8Mq3Feg1etIi0jhYcpAfD/zIsJRhJIQk1Hj9AnsBa46sYX/hfhKCE+plRaqgIkA6syyTH/b9QM/4nvSO710jN93B4oOsOLyCUmcpaRFpp30fhBB8su0Tvt37bb3nfDIhhhBZ8MT2pHtcd8KNZ9GaLATacpvs8vJJeILNdYt7UalAo0IgW0VOlnQ+ZIFlzC3EY3fhDg/moP0YwYZghrcaTnJYcoMcSiWEgKIiOeZHp6t7iReVSrb6VPyI8/lkMZSfLz/X6SAmBkLPXnmGU1EEkIKCQrPG6rbyV95f7MjbgcfnQa1Wo1Vp0ag0aDVaf02Sim7XBo3BX5RNq9YGCKaK/51eJ5syN7GncA8RxgjSItIa/Tg0ag2p4akcLjnM0gNLGZoylNSI0wcLCyE4UHSAtUfXUmgvbPC0crVKTVJoEmWuMtYdXUeuNZeLky4mLjiuyvGSkPgz50/WHVsHgjNajXySj9mbZrP84HIARqaOJNwYHvC+nPzwv2fqk17TVP1aqDG0STKzVB4vuuJS9KVWfHodvqD6FzCsDsloQNLp0JSVk5e7j+j4lgzpOJL4Wojn2u1QkjO98vNlK01DWqA1mkB3WVGRLIqaEEUAKSgoNEucXid7C/ayLWcbebY8ooKiCDYF45N8eIUXSZJwepzY3LaAZSeX3kcFKiHfoDVqDVq11n/TtHvspISmnNW6LWqVmtbhrTlWdoyf0n9ikHcQnaI6VSkiPD4Pm7M3syFzAzq1jrSItEYLdA0xhGDSmjhSeoQljiX0i+9H55jOAW4kj8/D78d+Z1PWJkINoWd0FXp8Hmb9Pou1R9eiVqm5v8/9DG89vFHmf7bQ2J3oi0rR2J14zSaEtvHT/CU1HJTyiTdFMsDQkRZWFRjdDe8+8nplq09Rkeym0jWf+k2NhSKAFBQUmhVeycuBogNszd5KZnkmoYbQSvVtaosQAp/wyUJJ8iIQJAQnnJUihVWRGJJIni2PXw7+gt1jp1dcrwA3UqmzlDVH1rA7fzcxlhjCjGGNPiedRkdqRCr5tnx+PvQzObYcLkq8iDBjGFa3ldUZq9mZt7NGLjiX18XMtTPZnL0ZrVrLtP7TuDjp4kY/hkZDktCVWtEXl4IAT0gdXV61xCd8HC45TEJIAgNbDiRCFyILFIcDoqJkodIQ83C5IDdXjvsJDZWtNRcATR6O/e6775KSkoLRaKRfv36nrfLr8Xh44YUXSE1NxWg00q1bt0otEZ5//nlUKlXAo337ho+SV2LHFRqLC/WzJQmJQ8WHWLJ3Cd/v+54SZwmp4anEWmLr7epQqVRo1VoMWgNmvRmL3tJk4qeCaHM04cZw1hxZw5qja/D45IrEGSUZfLfvO3YX7CYlLKXW4kcIQaG9sM7zijJHkRyazM68nSzZu4SdeTtZemApO/N20iqs1RnFj81t47mVz7E5ezN6jZ5/DPrHOS1+VG4PhrxiDAXFSFqdHJB8Fj47HsnDoZJDJIclMzR5qNx+RauFiAjZVXX0qByk7D59/7EzYrdDZiaUl8uZXheI+IEmtgDNnz+fhx9+mPfff59+/frx5ptvMnLkSPbu3Ut0dOUuxs888wxz587lo48+on379ixbtoyrrrqKdevW0aNHD/+4Tp068fPPP/ufaxuwDkFFZWK73Y7J1Hi+X4ULF/fxC5rmArkQCSHIKs9iW8429hftR426UQOSmxPhpnB0Gh0bjm3A6XESGRTJhswNeCQPaRFptRZ+XsnLy2tf5o/MP7g46WLu6nVXnYKDjVojaRFpZJZlsuzAMlSoaBPR5oyZVWWuMp5b8RzpxekE6YJ4dvCzdIzqWOv9Nxc0Vrvs8nK68ViCQFM/IS4Q+CQfHsmDJElyVpvw4pN8+IRsnaz4XyBoE96GAS0HVO7objbLWVSFhbKAqas1qLxcFlFer1zj5xwpYNhQNGkafL9+/ejTpw/vvPMOAJIkkZSUxP33388TTzxRaXx8fDxPP/009913n3/ZNddcg8lkYu7cuYBsAVq8eDHbtm2r87zOlEaXnZ1NSUkJ0dHRBAUFNfkvSYXzB0mSyMrKQqfT0bJly/P+s1VgL+DPnD/ZXbAbl89FQnBCs22ceSpWt5WlB5bK6ddxPc68wmlwep0cKjmEWqWmhakFkUGRtd6GT/Ix6/dZrD6y2r8sWB/MHT3vYGjy0Dp/ltw+Nzq17ozrF9oLeXbFsxwtO0qoIZTnhz7fpBWh64XPd9zlVYZQq04b6CwQeHwenF4nLp8Lj8/jFzOn3l5VKpUcwK/R+GPStCoteo0eo9boD+Q3aozotXqSQ5PP3CXeapUFTIsWsnWoJrE7QkBJiez20miappNBURHEx8tzbkDOiTR4t9vN5s2befLJJ/3L1Go1I0aMYP369VWu43K5KtVFMZlMrFmzJmDZ/v37iY+Px2g00r9/f2bMmEHLli2rnYvL5cLlcvmfl5WVVTsWIDZWrr9R0RBUofEQQpz3IuBU1Gr1eS9+ylxl/JX3F3/m/Um5q5w4SxwhhjrUGmkCJCHx66Ff+Wz7Z5S6SuUA3773M7xV3QN8jVojbSPa4hO+Olm+JCExe9NsVh9ZjVat5bbut7H84HIOlRzijd/fYPWR1dzb+946CauazCe7PJt/rPgHebY8Ik2RvDDshRr1TmuOqF1u9EWlaK12vEYDQi8LCklIOL1OnD4nLq8Lp9cp1yIS+LuhB2mDMAeZMeqMmLSymKnIeKvIXtNr9AF/dRpd/Vy8FotcgDA/Xy5cGBUlL6vu+iFJsuUoP1+2Il3AnowmE0AFBQX4fD5iYgIrdMbExLBnz54q1xk5ciSzZs1i8ODBpKam8ssvv7Bw4UJ8J6XS9evXjzlz5tCuXTuys7OZPn06gwYNYufOnQQHV+27njFjBtOnT6/x3FUqFXFxcURHR+PxnLmTsELd2J6znVxrLkNShjRJh+WmQq/X+3uznW84PA525+9mW+42CuwFxJhjiG9R93YTZ5v04nQ+2PQBewrla1SwPphydzlv/fEWHp+Hy9tcXudta9QaNNTe7SmE4N9b/83yg8tRq9T+gOPL21zOwt0L+fKvL9mUtYmpP07ltu63cWnrSxtUXGeUZPDciucochYRZ4njn8P+SbT5lBAGSULt9qDySfhMzbR5phBoym2o8guwO21YjSqczmI8Nvkar1Fr/JaaGHMMEUFyk1SzzoxZb8asM2PUGpvmh4tOJ8fv2GxybFCLFvLj1PAPr1cWPoWFskhqhEKE5xJN5gLLysoiISGBdevW0b9/f//yxx57jJUrV/LHH39UWic/P58777yT7777DpVKRWpqKiNGjOCTTz7B4XBUuZ+SkhKSk5OZNWtWtX2nqrIAJSUl1ciEptA4pBel88P+H3B4HAxvPZxe8b2aekoK9cDj87C/aD9bsreQXZ7tr7jcXDtsn4rVbWXun3NZmr4USUgYtUYmdprI2LZj+WTbJ3y//3sA7ux5J2Pbjj2rc5v751wW7JKbGP+9398rWaKOlB7hXxv+xb7CfQB0i+nGfX3ua5BmqPsL9/P8yucpd5eTHJrMC0NfINwkxxyp3B7Ubg8alxuNzYHa40UlSbjDgnFHNH2mkVfyYnPbsHlsOBzl6EvKMZU70OiNaC1yWYAIUwRhpjDMejNBuiC/2DkbVabrjNstu8XM5hPWoIrlublyE9OQkKbv0XUhu8AiIyPRaDTk5uYGLM/NzfW7mE4lKiqKxYsX43Q6KSwsJD4+nieeeILWrVtXu5+wsDDatm3LgQMHqh1jMBgwGC4cC0Nzp9BeyJoja9CoNEQGRbI1eyspYSl1ah+g0LT4JDmNd2vOVg6XHMasM5+xcnBzQhISvxz6hc+2f+ZvsDmo5SBu636b//P4t55/Q6/Rs2jPIj7a8hEen4erO1x9Vub3ze5v/OLn7l53V+mGaxnakpeHv8x3+75j7o65bM/dzv0/3s/kbpMZlTaqziJ0R+4O/m/1/+HwOmjboi3PDfwHYRhRl1rR2B2onS7UPgkhBJJeh9dkRCUE+uIy1D4JV4swhO7s3IJ8kg+7x47NY5PrRgkfGrUGs85MpMpCojqCsBANxpZRmM2hTWvNqS96vWwNslpla1BkJAQFyZafqhqaXsA0mQDS6/X06tWLX375hfHjxwNyAOgvv/zC1KlTT7uu0WgkISEBj8fDN998w4QJE6oda7VaSU9P55ZbbmnI6Ss0Ek6vk9VHVlNgL/BX591XtI+tOVsZ3mr4uXlBOs/w+Dy4fW5cPhcurwuXzyU/P/6/zW3D6rZi99ixe+wUO4rRqDW0DmvdoFWMG5sDRQd4f/P7fstJUkgSd/W6i64xXQPGqVQqpnSbgk6tY8GuBczZPge3z83EzhMbdX4/7P+Bz7Z/BuAXM9WhUWsY3348fRP68s6Gd9iZv5MPt3zI6iOreaDvA7VqywGwMXMjM9fOxCN56NaiE891uZ+QPDtqTxlIcg8rSafDF3TiFiMJiWJnCWodBBXmonLZkWKiUBsbNgZFEhIOj8MvdjySB5VKhVlnJlgfTNsWbYkMiiTMEEqYW425xAYmj1z/5ny5vqhUclZYhdVHp5OrLlfV0PQCpkmzwObPn8/kyZP54IMP6Nu3L2+++SYLFixgz549xMTEMGnSJBISEpgxYwYAf/zxB5mZmXTv3p3MzEyef/55Dh06xJYtWwgLCwNg2rRpjB07luTkZLKysnjuuefYtm0bu3btIiqqZk0Oa2NCU2g4JCGx+shqNhzbEFDy3+q2km/L58r2V5ISltK0k7wAcHqdZJVn+QWN0+PE6rHKNxSXTc50qWhk6ZP/yt2t5fU1ak1A24JgQ3CN+ks1F8pd5Xy+43OWHViGQGDSmpjYWXZ3ncn1seCvBczdIWekXtfxOm7ucnOjiPZfD/3Km3+86d/PLV1r/gNPEhJLDyzls+2f4fA60Kl13NjlRsa3G39my5zPx+r035i19V18wsfFYV15ttUUdBo9kl6HpNNWaV0QCA4XHybEFIIaNR6fG1V5OS41lEeY8Zr0IECFCq1Gi16t9wcOn/z3VGuVECJAdDt9TlSoCNIFYdFbiLXEyvWWTOGEGcOw6C3yNnw+OQ6moEC2mASdG5mHdUIIWQg1Ny/HhewCA7j++uvJz8/n2WefJScnh+7du7N06VJ/YPSRI0cCgkGdTifPPPMMBw8exGKxMGrUKD7//HO/+AE4duwYN9xwA4WFhURFRTFw4EB+//33GosfhaZjd/5utmRtISE4IcBSYNFbyLflsylrE/HB8RdEfZimwuPzsOLwCv7K+8svalSoAntqaXSYdeaAvkzng2VOEhLLDy7nP9v/Q7m7HIAhyUOY0n0KLUw1c79O6DQBnUbHp9s+5atdX+H2ubmt+20Nen7WHV3Hvzb8C4Cxbcdyc5ebTzkQCV1JGRqnG6FSg1qFUKv8wkSoVIyLGsRFAzvw9s5/s7ngTz7b/hnrDq/mwW53kxKaLFsJVCqOdxFB7fGidjhZevhn3jr4BQLB8BZ9eaT9raj0Rrynma9AkFGSQaQ5kqEpQwk1huL2unH6nHhKCnFLPhxRodiNGhweB1a3lXJ3ud96Y3fZ8fg8lTrACwQqVBi1Rsw6M63CWxEXHEeYMYwwYxihhtCqBZ3LJbd8KC29MFo+qFTNT/w0E5rUAtRcUSxAZ5/s8mz+t+9/AFUGZ3p8Hg4WH+TS1EvpFtvtbE/vgkAIwbqj61h3dB0pYSnnlNWmvuwv3M/7m99nf9F+AJJDk7mr1110ju5cp+19v/97Ptj8AQCj2ozib73+1iAB35uzN/Pi6hfxSl6GtxrO/X3vD9iuyutDV1iCvrRctsaALGKFOG6hE7JFAFnYCiFYWvg77x1biNXnQKvScFPsZdwYexk6te6Eu0RIfJn7Cx8eWwzAmPhh3Nf25jMek0BwpPQIYcYwhqYMrbp/mM0mW2RiYgKK8Qkh/K5Wp9d5wiJ5/H+f5PNbdsKMYTVzr5aXy+LH6ZRdXg0ZC3PwIPz8M3TuDD16NH16udcLu3fDli2ypWX48OYV+3OhW4AUFEB2ca0+shqbx1Zt4TSdRkeoMZTN2ZtJDks+K72RLjT+yv+LDZkbiA+Ov2DET5mrjM///Jyf0n/yu7tu7HIjo9NG1yvTZ3TaaHRqHe9ufJcfDvyAR/Jwb+976xX8vTNvJzPWzMAreRmYNJCpfaYGih+3B0NhCbpyO26LucZVi0eEjqBHQm/e3vcf1hds5bPsH1lVtoNHOtxOmiUZIUl8lrGY/x6Tf6Bc33IUt7a+9oxWLYEgs0zu5TY4eXD1zVPNZrm3VXa2LIQiIkCtRqVSyfV1tIb614iSJPmGm58vZ5+F17469mnZvBkmTZKLC4KcYdW5M/Tpc+JRRXeDBsVqlcXOhg2wcaP8v91+4vW2bWHqVBg3rukzwJoJigWoChQL0NnDK3n59dCvbM/ZfsbsICEEewv30ie+D0NShpwXbpfmQkZJBj/s/wG9Rl+5hst5iE/ysfzgcj7/83O/u2to8lCmdJ8i91xqIH479BtvbXgLSUgMTR7K3/v9vU4iaH/hfp757RkcXge943vz5IAnAyweapcbQ0EJGpsDT3BQnX7pCyFYmbeB2fvnUuqxolapuTbpchw+J99l/grAba2v5frk0TXaXmZZJka9kSHJQ4gPrkGtJ7dbtga1aCGnbzdUmrzHI1t9iotPtJBoSFauhNtvl0Vcq1by35ycyuOSkwMFUVpa/SwymZmwaZMsdjZskK09khQ4JiQEunWD7dvlRqcV87j3XrjuuqZ1jTUDC5AigKpAEUBnj81Zm/nt8G+0DGl55pLvyL/Yi53FjGs3jqTQpLMww/OfQnsh/9v/P8qcZRdEkPnugt18tPkjDhTLpTGSQ5O5u9fddIru1Cj7W3NkDa+tfw1JSAxIGsAj/R+plXXpcMlhnvr1KaxuK12iu/Ds4GcDCoNqbA4MBSWoPJ4GadRZ4i7jvf1fsCLvRC02FSrua3szYxMuqdE2MsszMWgMDE4eTGJoLSpCe73yjTo8XLaY1Dc+x2aTxY/dLouBhq49tGQJPPCALLKGDoWPPpJdX8eOycKk4rFnj9/16CcsDHr1OiGIunWr3m3m88kC52TBk5VVeVxSUqDIatdOFlllZfDZZ/Dhh7LwAIiNhbvvhptuapogcEUANU8UAXR2OFR8iB8O/IBZZ65xkCnI1XiTw5IZkzbmnEqrbo7YPXZ+3P8jh0sO0yaizTlTmLC2SEJiU9YmFu1ZxF/5fwEQpAvipi43MarNqEavS/T7sd95Zd0reCUv/RL68djFj9Xos5tZlsmTvz5JibOEdi3a8cLQFwJ+KGjLbegLikGAz9ywMSfr8rfw9r7PKfWUM6397VwS2//MKwE51hxUahWDWw4mOSy59jv2+U4EKMfG1s1KUdHrKi9P/r8ujULPxNy58MQT8vavvBLeeqv6ysqlpbJLqkIQbd0qW4pORqeDLl1k4dK3r2ytqhA8mzfLLq6T0WigUyd5fO/e8t+4uNPP2W6HefPg/fdPWKkiIuDOO2HKFFkkni2KiuT5tmjY+m6KAKonigBqfIodxXy//3tKnaW0DK2+T1tVuH1uDpUc4oo2VzTar/YLgZPdjzXp9H0u4va5WXF4Bd/u/ZajZUcB0Kq1DEsZxs1db65Tp/S6sjlrMy+teQmP5KFnXE+eHPDkaVu85NnyeOKXJyiwF9AqrBUvXvIiFv3xqr5CoCspx1BYiqTVyO0lTqIiW0pF/W76Lp8bu89BuD60RuNzbblIQmJw8mBahbeq+44rBIzJJIug2lgovF45vb2wEIzGhg9GFgLeeQdmzpSf33ILvPhi7axLHg/89dcJQbRpk1yv53RYLCcsRr17Q8+eNW9i6vUGxv24XPDVVzB7NmRkyMtCQmQRdOedDW6VAeTzdvCgfKwbNsDvv8Mjj8hxSQ2IIoDqiSKAGhe3z82yA8vYW7iXtIi0OlkdssqzMGqNXN3haoINVfd4U6geIQR/ZP7B2iNrSQpJqpH78VzC6rbyw/4f+N/+/1HiLAFki8/lqZcztu3YJqsqvj1nO/9c/U/cPjddo7vyzOBnqgw4L3IU8eQvT5JtzSYhOIEZw2ecCPyXJPQl5egLS/AZDEiGQEuSJCQOlhwEINYce0I0NTL59nzckptBLQfRJqJNw2y0tFS2jMTEyFacM+F0ykKivLxx2j0IAf/8J3wgZ/jxwAPw2GP1ty4JAUeOBAoiu10WORWCp0OHurnwyspkAVRV7y+vF779Ft5+G/bLGZCYTHDzzbJ7rJquDDXC5YIdO04Ink2bZFF6MjfdJFvSGhBFANUTRQA1HkII1h5dy+9Hf6dVeKs61/SRhMTewr1cnHQxA1sObOBZnv/syt/F8vTlcq+j8yijLteay5J9S1h+cDlOrxOASFMkY9uNZWTqSIJ0TV/w7q+8v3hh1Qs4vA46Rnbk2SHPBsyrzFXGU78+xZHSI0Sbo5k5fKa/i7vK60NXVIK+1Io3yIio4gafUZpBjCWGKHMUewr24PP5iLXENmr9rEJHIQ6Pg4HJA2nbom3DbtxqlYN7Y2Orr9YshCx6cnNl60pjVHX2emWxM3++/Py55+Bvf2vYfTQ0Npv812yWg8Cry36TJFi6FP71L1m0gCyWJkyA++6DljWw0hcXyyKnQvBs3y6LoJMxGORYpz59oH17uPxyaNNAYvk4igCqJ4oAajz2FOxh2YFlRAZF1ju1tcRZQrm7nPHtx9csy0QBgKOlR/nhwA+oUTdIQ8zmQHpROgv3LGTt0bVIQs6ESQlL4ar2VzGo5aBm597bU7CH6SunY/PYaNuiLc8PeR6L3oLdY+eZ357hQNEBIowRzBg+g7hgOa6jIs1da7XjMQdVmeZe6CjEJ3yMaD2CWEss2eXZ/Jn7J4eKD2HWm4k2R9fbLXYqxc5iyl3lDGg5gA5RHRp0234cDtm6ExMjx4ycLG58vhMp7o1V1dnplIXA0qWyFebVV+H66xt+Pw2JwyGLwYQE+bwcOSIHRJ/OJSgErFghC6ENG+RlGg2MHw/33y9nrlWMO3z4hLVqw4YTFqSTiYg4EZDduzd07XoipksJgm6eKAKocci15vLdvu/wSb4GEywHig7QJqINo9IaP5D1fKDIUcQP+3+g2FF8zmd8CSHYkr2FhXsWsiNvh39595juXNX+KrrHdm/WpRIOFB3guRXPUe4up3V4a54e+DSv//46u/J3EWII4aVLXvLHx9Ukzd3hdZBTnsPglMEBQsQreUkvSmd77nYK7AXEBMU0mNu4xFlCqauUixL70zm6U+Oeb5dLtmhERckNPjUaeVl+vhwvVJWLpyGwWuHWW2HdOvnm/d57MHJkw++nIXG7ZRdafLycbQay+yk7u+b9wH7/XQ7sXrVKfq5SycetVsvCJz+/8jqpqYGCJzW1+n0pAqh5ogighsfusfPD/h84Wnq04eIDAJfXxZHSI4xqO4r2ke0bbLvnIw6Pg2XpyzhQdKDOsVfNAY/Pw6qMVSzeu5iMUjmAU6PSMKjlIMa3H0/r8NZNPMOac6jkEM/+9iylrlK0ai1eyUuQLogXh71IaoRcFFRjd2LIL0bl9eI1m6q8oVTE/XSO7szAlgOrfG+tLit/5f/F7oLduL3uereVKXOVUeQool9CP7rGdj07YtPjkV1d4eFyTFB+fuNUda6gsFAOct6+XRZYn3wCAwY0/H4akopSArGxgdYyr1cOeBaidlaybdtki9CyZYHL9XrZonOy4KlNRpcigJonigBqWHySjxWHV7Ale0ujZBsdKztGsCGYq9pfhVlfw6yICwyf5OO3w7/J70F4m3OyfIDVbWVZ+jK+2/cdRQ65lolJa+Ky1Mu4su2V1VcabiSEEBQ6CrG6rSSFJNXZAnm09Cj/+O0fFDmLMGgMTB86nY5RHQHQWu3o84vOmOZeEfczotUIgvSnv7lVuMUOlxwmSBdUJ7dYRYPiPgl96B7X/eyK6Yo0ebVatgLVJDi6LmRmwo03woED8k163jz5ht+cqTg3kZFyHaVTRWFJiVyjKCys9oJx9274+usTbq2uXeUsu7qiCKDmiSKAGpZt2dv45dAvJIYkNkoQqk/ysa9oH4OTB3NR4kUNvv1zHSEEG7M2sjpjdaO9B1WxMWsji/csxuVznXlwDThaehSHV66dEmGKYGxbObD5bGU5nUyRo4g8Wx7hxnCCdEHk2HJIDU+tsxDILs9m0d5FDE0eKosfIdCVyTV+hKZymvvJFDoK8UpeRrQe4Y8XOhNeycvB4oNsz9lOvj2f6KDoGsfk2dw2cm259IrvRa/4Xk1jSRRCtmg0ViPTAwfghhvkYoPx8fDf/zZ4sG6DI8SJQOfY2Kozxnw+OHpUdh02lnCsKc1AADWvyECF844jpUdYf2y9/0bRGGjUGmLMMWzP2U6rsFbEWGIaZT/nKvsK9/H70d+JCoo6a+Lnh/0/8OGWD/0ByQ1Fcmgy49uPZ3DLwU1ixSp1lpJjyyHYEMzFSRfTKboTXsnLTwd+4mDxQVqHt66TIIgLjuPe3vfKT86Q5n4yDq+DUmcpg1MG11j8gFwLqW2LtiQEJ8husfzdFDoKSQhOOK1bzO6xk2PLoUdcD3rG9Ww6N6pK1Xji588/5fTsoiI5huW//5UDiZs7FcUjo6OrT5fXaGQ31ZEjshhq6MrY5xiKAFJoNEqdpazOWI1H8pAYUoty+HUgwhTB/qL9bMnewsg2I8/Z+JaGJqs8i9UZqzFoDYSbGr/onyQkPv/zc77Z/Q0Aw1sNp39izSoIn4lgQzDtW7RvksDmclc52dZsgnRB9I7rTeeYzv7UdIARqSP4Kf0nDpUconVY67rP0edDX3g8zd1kROiqv0RLQiKzPJPO0Z1p16JdnXZn1pvpm9CXlqEt+TNHzhYz6ozEmGMqfYccXgdZ1iy6x3Snd3zv8zPpYN06OeDZapVdPHPnNnil4kahtFQO0I6NPbMwtFjkGklWqxw7dQGjCCCFRsHj87DmyBqyrdmkRaSdlX0mBieyp2APbSLakNbi7OyzOVPiLGHFoRVYPVZSw1MbfX8en4d/bfgXKzNWAnBj5xu5vtP1zToT60zYPXZ/X6uuMV3pGtO1SgtjtDma4a2Gsyx9GYdLDtepCrLK7cFQVIq23IY3yITQnl5gHCs7RnxwPL3ietVbjMRaYolqHcXB4oP8mfsnB0sOEmWKItQo3yCdXieZ5Zl0je5Kn4Q+za6sQIOwbBncc4/sHurfHz79tOndRDXBZpMLPsbF1axtiEolu52sVjmovLEsaecA5+GnWKGpEUKwOXszu/J30Sqs1Vmzxph0JvQaPZuyN5EYknjeVTeuDU6vk9UZq8myZp0VAWp1W5mxZgY78nagUWmY2mcqw1sPR+XxApzWktEcqbjhq1HTIbIDXWO6Eh8cf1oxFxccx/BWw/kp/SeOlB6peYsXIdDaHOiKStG4PHgsZ+7mXugoRK/V0zehb4MF/mvUGtJapBEfHM+u/F3syt9FUXERkUGR5Npz6RTVib6Jfc/JAPozsmABTJsmu4VGjpRbRNQnwPds4XDIc05IqF1mV1CQHAhdVFR9ccQLAMVPoNDgHCg6wIZjG4i1xJ6211FjkBCcwLHSY/yV99dZ3W9zwif5WH90PbsLdp8VAVpgL+DJX55kR94OTFoT/xj8D4a3Hg4+H4b8Ioy5hWjszkadQ0Ph8ro4VHyIzLJMUsNTGd9hPCPbjCQhJKFGlqyk0CSGtx6OTqPjWNmxM45Xuz3o84owZuej8kl4QsxnFD8VcT89Y3s2SgFQs95Mn4Q+XJF2Ba0jWlPiLKFDZAcuSryoftWk//hDLprX3PjwQ3joIVlITJggPz8XxI/LJT9iY2tvqVKpZOGj1Vau1nwBcW79LFNo9hwtPcrqI6vRaXRN0mJBo9YQGRTJ1pytpISnBMRpXChsy9nG1pyttAxp2ajtDwAOlxxm+srpFDoKCTeG8+yQZ/3uNl2pFa3NgVCrMebk4w4PxRNqaZx6LfXE4/OQbc3G5XWREp5C99jupISl1Ek8poSlcEmrS1ievpys8qyqRYokySnuxWWoXR685jO7vOBE3E+nqE6NV3X5ONHmaC5pdQnZ5dlEmiPr92Nm/nx4+GH5/xtukNtINLV7qaQEpk+XrT8gNwF99tlm+fmshNcrFzqMial7HI/RKLvCcnNr5jo7DzkH3mmFcwEhBHsK9vDDgR+wuqwkBDdd1kRkUCSlrlK2ZG9p8Cyk5s7+wv2sP7aeCFNEo9dE2p67nSd+eYJCRyGJIYm8eumrfvGjcbjQl5bjMxjwWoKQtDoMhSXo84v9brHmgFfycqzsGIdKDhFljmJMuzFc2e7KOmdzVdAmog3DWg3DK7zkWgO7fKtdbgx5xRhzC0GAJ9RSI/EDkFmeSXxw/FkLQtaoNSSGJlbZsLXGLFkiu5cq+O9/YfhwWL26/hOsKz/9BJdcIosflQqeeEIWZeeC+PH55EKHkZGV24LUltBQWQg5HA03v3OIc+DdVmju+CQfm7I28dOBn1AJFa3CWzV54GticCK78ndxuORwk87jbJJjzWFVxio0Kg0tTI2bubLy8Eqmr5yO3WOnY2RHXh7+MtHmaPlFnw9dUSn4hD+FWzLo8ASZ0Jdam4VLzCf5yCrP4kDRAcKMYYxKG8X49uNp26JtgwX4to9sz9Dkodi9dvJt+SBJ6EqtGLPz0VpteMxBp63vcypFziK0ai194vucOwU/ly+Xe0hJklxU8OuvITlZLjI4cSI8/bRsyThbFBXB1KlyplduLrRuDYsWyXNsiI7uRUVyLR6HQ37e0AghZ3yFh8sCqL5z1uvlbTXWfJs5igBSqBcur4tVGatYlbGKMGNYrWqRnA6Pz8Pao2vJs+XVaX2z3oxapWZT1iZc3vPbx21z2zhQdIAVh1dQ7ipvVOubEIJvdn/D67+/jlfycnHSxbww7IWA3lK6UitauwOv+RSrgUaNJ8SM2uXGmFuArvR4l++zjNVtZV/RPkw6E5e3uZyrOlxFh6gOjeIu7BTdicEtB2MvK8RxOB1DXiFCrcYbbK6ymWl1OL1OShwl9IzrRULIOVCTBmDNGrjrLtldc9VVMHOmnF21fDlMniyPmTMHLr30ROPNxuR//4Nhw2TBo1bLGV8//SRXNa4vQsgutZAQuQ4PyM/LyuS+XA1BxT7OVOuntoSGyg1Sz6YQbSYoMUAKdabcVc7KjJXszt9NUkhSg/wqlYTEqoxVzN0xlzxbHkG6IKb1n0bv+N613lZicCLpxensyt9Fj7ge9Z5bc6KiLk1GSQYZpRmUOkvRqDWkhKU0mvXNJ/n4aMtH/HDgBwDGtRvHrd1vDXAVaexO9CVleI2Gqt0JKhVeS9BxN1AhKpcLT3joWcsSs7qtZJZn0je+L30T+zZ+YUifj27aBDTq1mwq/gN1i3hCDLWrXC0JiWPlx+gY2ZGOjRz302Bs3ChbWVwuOavqjTdO3LDNZnjpJbj8cnjkEbmr+NVXyzE4jz12+m7ldaGgAJ56Cr7/Xn7eti3MmgU9GvCaUFoqZ1bFxp6wqtjtct8yq1V+GAyyu6muwqWs7MQ+GjJ1XauVrUnHjsnn/lxwAzYQSiuMKlBaYZyZPFseKw6vIKMkg9bhrRvk1/PW7K3M2T6HQyWHAPzNIVWomNRtEle3v7rWN/cKC9LVHa4+K4UAG5NSZynZ1mwOFx/maNlRfwPNcGM4oYbQRo0JcXldvLb+Nf7I/AMVKm7rcRvj2o0LHOTzYcopRO104bWcWViovD60Nge+ICPuiFB8QY2beVPmKiPbmk2fhD4MSBrQ+LVsbDb55ltejmQysq14DxsyNxBtjq5V+45jZceIMEUwovUILLUUT03Cjh1yNlVZGQwZItfTqS7ItqxMDkT+8kv5eZs28OabDSNOhIDFi+Ef/5DdUhqN7P76+98bNui3vFwWEfHxVYs3l0sWQ6WlJ6wsFWKoptczq1UWJgkJDS8QQbbEHj0qN5Y9W8HpzaAVhiKAqkARQKfncMlhVh5eSaGjkNZhret94z1QdIDPtn/G9tztAATpgri2w7Vc3uZyPtv+GcvS5S7Eg1oO4oG+D9QqG0UIwb7CffSM68mwVsOaPDapNgghKHGWkG3N5mDxQTLLMyl3lqPT6Ag3yaLnbNRYKnOV8c9V/2Rv4V50ah0PXfQQA1sOrDROX1SKvqAYT4il5hf24zVwUKtwR4TJaeCN8B6VucrIsebQL7Ef/RP7N24Asdcr33ALC+XnFjnzTRISm7M2sylrE3GWuBpZn4qdxTi9Tka0HnFuuL727YNrrpFvbv36yQ1Ea3LD/vln2fqTmyvf6O+7T05Nr6tQyc2VA5t/+kl+3rGjbIXq3Llu26sOm00WWgkJsmXrdEiSHGtjs8nCz+mUhZPJdHqLjsMhFyxMSGhccVJeLoug4OCz0yJDEUDNE0UAVY0Qgl35u1iVsQpJSCSFJNVLUORYc5j751xWHVkFyBaf0Wmjua7jdQGNGX888CMfbv4Qn/DROrw1Tw98uladv8td5RQ6ChmUPAij1ogkJIQQSEKS/+ek/4XAK3nxCR8+yYckJHzCJy876XmQNohQYyhmvRmT1oRJZyJIF4RJa6pXurAQgiJHEdnWbNKL0sm2ZmN1WdFr9UQYIwg2BJ/VNh/Z5dlMXzWdrPIsLHoLTw98mk7RnSqN09idGHPy8el0CH3tzfNqlxuNy4071NLgLrESZwn5tnz6J/Wnb0LfxhM/Qsg3t/x8+a/FIrtDTsIn+diQtYFt2duID47HpK1eHFQUYxyYNJAusV1qPo916+QWDlOmQN++dTyYOlDhysrNhW7d5NT32tywi4vlNPSFC+XnHTrI1qDaiBYh5Myu6dNli4tOJ1t87ruv0ntRbxwOOb4nIUGO/akNFWnsZWXyZ8XjkcXeqS6oCutRQoJcuLAxEUJu/lpa2vj7AkUANVcUAVQZr+RlU9Ymfj/2OxadpV4NR0udpSzYtYAfD/zod3ENSR7CTV1uqna7O/N28vLalyl1lRJqCOXxAY/TObrmF8ZjZcewe6oI8lMBp3wDVCoVKlSoVCrUqEEFatTy8+Piwyt5cftOBDdq1BoMGgMGrQGjxkioMZRwUzhmnVkWRjoTJq0skoxaY4BwlIREob2QrPIs0ovSybXlYnVbMWlNhJvCCdYHN4nlan/hfl5Y9QKlrlKig6J5bshzJIUmVRqn8vow5tbc9VUdfpeY2SS7xGqRIVUdxY5iCh2FfvHTaOLR45Ev6IWF8q9nS/VWMI/Pwx/H/uDPvD9JDK46xVwSEgdLDtIxsiMDkwfW3F33118wfvwJV8vkyfDkk43v1sjMlMXPsWPQvj189VXdb2w//CBbbwoLZQvJgw/Krqszxb1kZsrr/fqr/LxbN3j9dVlINTQulyyA4uLqX0nZ6Qx0kalUJ2KFysvlmJ/6prvXFLtdbpR6JqtUfZEkWfAmJCgCqLmhCKBAnF4na46sYVvONmLMMXUucOj0Olm8ZzGL9izC4ZXrTvSI7cHkbpNpHd76jOvn2fJ4cfWLHCo5hEal4W+9/sYVba6o01waGq/kxeV14fa5cflO/BVC+AWUXquXRZLGQIgxhDBjGEHaIA6XHCbPnofdbSdIFyTX8NGZm9RdtzFzI6+sewWXz0Xr8NY8O/hZIkxVX6j0RaXoC0vwBDeA+0oItFYHaOrvEityFFHkKGJAywH0ju/dOOJHCPkmlZ8v3xCDg2t043D73Kw/up6/8v+qsmDl0bKjhBvDuTT10oAMu9OSlwejR8u/4hMSZEEA8g10xgy47LLaHl3NyM+Xxc/Bg9CqlWzBqciEqiuFhbKY+UEOuKdrV9ka1K6Kpq9CwBdfwAsvnAg2fuQROQNN2whxXh5P4wiTCheZ1Spv3+WSg5Ojo8+O+KkgJ0c+/43VIsPtlo8xJEQu5NjARRgVAVRPFAF0glJnKSszVrKnYA/Jocl1yprxSl6WH1zOlzu/pNhZDECb8DZM7jaZbrHdarUtl9fFvzb8i9VH5CJql6dezp0972z2/Yl8ks8vjNw+t18seYUXs85MhCmi8TOSzoAkJMpd5aw5uoaPtnyEJCR6xPbg8QGPVzs3jc2BMacAn0HfoG6r+rrECu2FlLhKGNRyED3jejaOmHS5TtR90WrlGJBa7MfpdbLuyDr2FMrfLZ1a/gwXO4txeB2MaDWCxNDEGm7MCddeC1u3yrVtvvsOdu6Exx+XXVMAY8fCP/8JUTV3H5+R4mK47jrYvVsWXYsWyX8bAiHg22/lWkElJbIL69FHZWFTEaNy5Ii8bM0a+XnPnnKGV1oj9b/z+WRLTVRU4woTj0d+T4OCzk48zsk4nfJ51WobviVIhbuvoohjIxybIoDqiSKAZHKsOfx2+DcyyzJpHda61iJDCMH6Y+v5/M/PySyXf43GmmO5pdstDEgaUOdf5BW1aD7/83MEgo6RHXl84OOEG8/tLK/GQhISZa4yihxFfpdQsbPY/7zIUUSRU/7fJ3z+9Ya3Gs59fe6r1v1S4fpSudz4zFXHsuTYcvD6vMRYYvw3+JpSV5dYvi2fcnc5g5MH0z22e8OLH0mSb4IFBfKv2eDgOlsaHB4HazLWsL94PymhKfiEj2NlxxiQNICusV1rthEhZBfR4sVy7MZ338kiCGSLwqxZ8MEH8s07LEyOs5kwof437/JyuZjhtm2yGFi4ULYANTS5uXKA9M8/y8979ZKPafVqOZ3ebpdv1I8/Drff3niCQZJkIRYRIVt/zud08YIC2RLUUO6piu+MwSB/VoKDG008KgKonigCCNKL0lmZsZIyVxkpoSm1Dhz9K+8v5myfw97CvQCEGEKY2GkiI1NHNpi1ZlPWJl5b/xp2j51IUyRPDXqKNhFtGmTb5xpHS4+yu2C3LGpOETfFzkBhcybCjGGMSRvDdR2vO6140BeWoC8qrdb1ZXPbKHYWE22J5ljpMYxaIzGWGDSqWnyWjrvEhEaFu0WYXEDwNHPKs+Vh89gYmjKULtFdGl782Gyye6CsTI6TaICUZLvbzqojqzhYfBCA9i3aMzhlcM3jft54A157TRZhX3wBAwZUHrNjh9yOYudO+fmAAfDKK5CSUrdJOxxw001yg9PwcPjmm6rdUw1FRXDzc8/JwutkLrpIPv7GEF8n77+4WC4aGBfXOK615oTHAxkZ8netvp9xl0v+3oSGyuKnkfuOKQKonlzIAkgSEjtyd7Dm6BrUqEkIrlkX7AqOlh5lzvY5bMzaCIBBY2B8+/Fc1f6qRnHxHCs7xourXySzPBO9Rs/9fe5nSMqQBt9Pc2Z7znaeW/ncafueqVARYgghwhThf4Qbwys9DzOG1Uignsn1JRCkF6fTLaYbfRP6klGawc7cnWRbswkxhBAZFImKmn+u1E4XGrfcNNQbYsFrMlT6pZ9jzcHlczE0ZWitAuRrhMcj3wCLiuSbYXBwg1oArC4rKzNW4vK6GNF6BCHGGl53vvsO7r5b/v+VV2RRUh1eL3z0kSwWnE7ZajJtmlyAsDY3dJcLbrsNVqyQz8NXX0GXWmSp1YfMTHnOq1bJ7qGnn4ZJkxrfGlNSIu8vPr7hs8maK8XF8vkOD6+7tcZqlS2PUVHyds6CO08RQPXkQhVAHp+HDZkb2JC5gVBDaK1SzUF2PUz9cSoOrwO1Ss1lqZcxsdPEaoNnGwqr28qs9bPYlL0JgKvbX80tXW85K80im5pSZyl/X/p3ipxFtAlvQ6vwVlWKmzBjWIMV/lN5fRhzClC5PdW6vgrsBajUKka1GUWoUe5W7fQ6SS9K56+8vyhwFNDC1IIwY1jNhZBPQuN0ofb58JmMeELM+ILkLurZ5dl4hIdhKcPoGNWxQY4TqBzkXEVqe0Nhc9tw+9w1L9i5fbscfOx0wh13yKnfNeHwYdmltHat/LxzZzlbqibp5h6PLLiWLpUtA//9b8O0kqgNQsgxP6mpsiBpbMrK5MD2hISGj4lpzvh8ciyQxyN/7mu7bmmp/BmpcHmdJRQBVE8uRAFk99hZnbGaHbk7iAuOC6jDU1Pe3/Q+Pxz4gVZhrXj04kdJDKlhAGcD4JN8zN0xl292fwNAz7ieTOs/rVYVd881hBC8uPpFNmRtICkkiVmXzapXDaIa7hR9YQmGojLc1WRoeSUvGaUZDE4eXGXNIJvbxt6Cvewu2E2Zq4zooOiaZzoBSBIapxu1x4NkMHBUXY4rSM+QtpfSPrJ9fY4uEKfzRJCzXi9bAJpLIc3sbDnjKzdX7mo+Z07tfl2fWi9Ho5GDix9+uHqXhyTJNXUWLpTdGJ99BoMGNcjhNFusVvk9T0iQ3/8LjbIyWQSFhdXcylaR1h8eLlt+zrLFrDb37/M4ikuhprh9bpanL+fP3D9JDkuuk/gpsBfw00G56uodPe44q+IH5Do8k7tNZlr/aeg1erZkb2Ha8mkcLT16VudxNvl+//dsyNqAVq1lWv9pjS9+kF1f+hIrHnP1ZfxzrDkkhSaR1qLqTByz3kzP+J6MShtFt5huWD1WDpUc8pdGOCNqNb4gI54QC9nWbIIKy7hM2472ooV88a0vPp8sfI4elcVPcHCtM7waFbv9RDfzdu1g9uzauxZUKrj+etmNNXasfMyzZ8OIEScyqk5GCDktfeFC2V32/vvnv/ip6JAeG3thih+QLT8hIbIQPBMV1lK3W46Tiotr9u7CJhdA7777LikpKRiNRvr168eG03QF9ng8vPDCC6SmpmI0GunWrRtLly6t1zYVYHf+bvYX7ic1PLXKomw14evdX+OVvHSK6kSXmLMUD1AFg5MH8/Lwl4kMiiSrPItpy6exMXNjk82nsThUcohPt30KwK3db6VVeCMGgB5H5fGiLy5D0qgQ1cSM2D12hBB0i+l2xv5w4aZwLm55MVekXUG7Fu0osBVwpPRIQIHJ05FpzYIgE307j6RVZBu5Dk5Ghpy9YrfLF+TaYrPJxfyysuRfvOHhzSvgtcIKs2OHnKEzZ0793AvR0bKY+fRT+UZ/+LAsjB55RI57Afk8Tp8ut7VQq+Ff/2q8mkLNBadTjnWKjT2r7ptmh1otf84kSY4hqw6f74SlNDFRTnE/B7LkmnSG8+fP5+GHH+a5555jy5YtdOvWjZEjR5KXl1fl+GeeeYYPPviAt99+m127dnH33Xdz1VVXsXXr1jpv80KnxFnC5uzNNQ5+rYoCewE/pcvWnxs639CQ06sTqRGpzLpsFh2jOuLwOvi/1f/Hgr8WcL54e11eF6+tew2P5KF3fG/GpI1p/J0Kga60HI3Dhc9UtUgWCLKsWaRFptXKAhhtjmZIyhBGthlJYmgimeWZZJZn4pWqv+BmlmVi0BgY3HIwKeEpcmxGeLjsmikqkoVQVpb8y7Um77vHI1tUjhyRf/mHhTVO08n68uqrcnFAvR7+/W9o2bJhtnvZZbI1aPJk+fmXX8LQofC//8lB0x99dGL/48ZVt5XzA7db/gzExMiZSxc6ZrN8HqqzAjkcshs1IkIWP7WNF2pCmjQGqF+/fvTp04d33nkHAEmSSEpK4v777+eJJ56oND4+Pp6nn36a++67z7/smmuuwWQyMXfu3DptsyoulBggIQQrD69kY9ZG2rVoV+eU4Q82f8D3+7+nY1RHZlwyo9k0HPX4PHy89WN+PPAjAMmhyVzZ7kqGJA9pkO71TcXsjbNZmr6UcGM4/7r8X/4g48ZEY7VjyinEa9JXa/0pdBQiEFzR5oqaB/KeglfyklGSwV/5f5FVloVZbyYqKMpfM0ogOFZ2jCBdEINaDqq+UKDHI1uBJEk24YeFyRfyU3+V1ifI2eORg4fDw+VMpMYWTN98Aw88IP//5ptyAcLGYMMGubjggQOBy//5Tzn763zG65XjXmJi5GJ9zeRa1uQ4HPKPCqPxxPej4rujUsmWxNDQZmH1OSdigNxuN5s3b2bEiBEnJqNWM2LECNavX1/lOi6XC+MpUfgmk4k1x33WddlmU2Cz2bDZbAEWCbfbjc1mw+VyVTlWkk6kOHs8Hmw2G85T4h1qM9Zut7Mvex87cnYQHxyPSqXC5/XhtDtxOQPn4HK4cNqd+HwnaslUjM0uyg6w/rhdbnms96Sxvmq263RVP9YROLZiu16Pt8ZjVZKKe3rfw72978WgNpCRn8Hba97m9iW38+XOLyl1luJxe3DanXjcHv/6kiThtDtx2gPPWVVjhRD+sSe/n/UdC/jHnvx+rj64mqW7l4IXHrrooRMZVlWM9Xq8OO1O3K5Al1JV7+fpxrpKrWjyi/2uL5/Xh9PhwnXSWK/kJbc0j9TgtoToT1x0vF4vNrsdhyMwvsfhcGCz2/F6A99Pl9NNnCmekakjGdpqKEatkT25e8kszsLr9criRx/EgMQBhOsisJ+yXafTic1uxwPyBTkkBMlqxbZ7N7Y9e+Rfqsf36SwpwXbgAJ6DB2WhFB6OpNVis9uxVfTSqjgPLhc2ux23+8Qxi0WLsL39NrYXXoDBg+V0cEmqeqwQ/u1W+t7b7ZW/96eO3bgR9yOPYANc99wTIH4qxlb63tvtla8RNRnbty8sW4b9vvuwabX4QI7/ue22at9P+/H3M+AzVcv3vrZjq33vPZ46jZW8Xmw5OdhMpoAWFy6XC5vNFvB+SpLkv96eTFVjhRCnvebXZmxN7g8NcS+x2+3YbLYT76fJhMdiwZafL4/1emWXl9GIvUULbHo9vpPm6/V6sdlsVb+fJ2+3iWkyAVRQUIDP5yMmJrD5ZUxMDDk5OVWuM3LkSGbNmsX+/fuRJInly5ezcOFCsrOz67xNkD+0ZWVlAY/GxGKxYLFYKCgo8C979dVXsVgsTJ06NWBsdHQ0FouFI0eO+Je9++67WCwWbr/99oCxKSkpWCwWdu/e7V82Z84cLBYLEydODBjbsWNH2ie059DuQ/6g59U/rmZCzwn8373/FzD24eseZkLPCezatMu/bOOKjUzoOYEnJj+BR/LQMbIjXaO78uTNTzKh5wS2rNniH/vn738yoecEHpv4WMB2p985nQk9J7D+5xPidO/2vUzoOYEHxj8QMHbmAzOZ0HMCK79b6V+WsS+DCT0ncNfIuwLGznpsFhN6TmDZgmUAXN7mcl7q8RK8BKpZKkpdpXyx8wtuW3Ib9997PxN6TuC7z7/zr1+cX8yEnhO4oW+gO+/fM//NhJ4T+ObtL1AfFyu2chsTek5gQs8JAUJu7ptzmdBzAnPfnOtf5vP6/GNt5ScunF998BUTek7g3zP/HbC/G/rewISeEyjOl9uH5Nvyeetfb8FL0HJVS7rHdvePvXXIrUzoOYGcIyc+58sWLGNCzwnMemxWwHbvGnkXE3pOIGNfhn/Zyu9WMqHnBGY+MDNg7APjHuC6fjeyb/s+v+tr7eotjLv8bp45abvZ1mz+9dQXXDRgHL9WpFcDy1aswJKWxuBrrgnY7oiJE7GkpfG/iuq+wOo//sCSlkafUaMwaA20j2zP6LTRfPn6T9w2/mm+XLKEYH0wQ5OHkptRhCUtjY5DhwZsd+K992JJS2POggXyAo2G3Tk5WPr1I2XgQDmwOSMD8vK4/ZZbsLRty7uLFvmDnI9kZmJJSyO6a2AV5qnPPIMlLY1X33vPv6xg7lwsgAVkd9uDD8IVV/D4ffdhSUtj+qwT58fucGBJS8OSlhZwM54+axaWtDQef/HFgP1VjC0oKpJjkm6/nVc9HizA1IrYnONEd+2KJS2NIxW9v4B358zBkpbG7dOmBYxN6dcPS1oau/fv9y+bs2ABlrQ0Jt5774mBRiMdv/0Wi9fLlhkz4P77AZi/ZAmWtDSuvPXWgO32GTUKS1oaq//4w7/sfz//jCUtjRGnXHsGX3MNlrQ0lq1Y4V/269q1WNLS6H/llQFjr7jlFixpaSz68Uf/st+3bMGSlka3k37oAlxz551Y0tKYV9FNHtixezeWtDTSBg4MGHvLAw9gSUvjw3nz5AVCkL5jB5Z+/Ujo1SvAknHXXXdhsVh46623/Muys7OxWCyEndI1/eGHH8ZisfDSSy/5l5WWlvqv+ScLuaeffhqLxcLTTz/tX+b1ev1jS0tL/ctfeuklLBYLDz/8cMD+wsLCsFgs/nsgwFtvvYXFYuGuuwKviwkJCVgsFtLT0/3LPvzwQywWC7fcckvA2LS0NCwWCzt27PAvm/fjj1j69uWaW2+VLT8tWkBiIt3698disfD777/7xy5atAiLxcIVVwT2aux/fOyvFQ1rm5imt1fVgrfeeou0tDTat2+PXq9n6tSp3HrrrajraXabMWMGoaGh/kdSUuWO1+cbHkm+gceY697VHeQYIpCtP83F9VUVZp0ZAJPOxCP9H6FNRBs8koes8iwAlqcvZ2vO1tPHCR1/TVtuR1tcVrcg2zrik3zM+n2WP0C4Js1jGwLV8WP0mfTVugMqAp+N2oZ3AQXpg/wp8u0j2zEkZQhxwXF125hKJburhJBdXhXXjbpcP9LTYfPmE8+fekoOlt25Eypu1oWFdZvnydhsclxOYeGJBqNn+3t2ihg87xBCDvg2m5t6Js2biia/FWUBYmObV4JAHWiyGCC3201QUBBff/0148eP9y+fPHkyJSUlfPvtt9Wu63Q6KSwsJD4+nieeeIL//e9//PXXX3XepsvlCjAXlpWVkZSU1GgxQBVm06CgIL9ocLvdeDwetFothpNKhVeMNZlMfqHn8Xhwu91oNJoAl2BNx1rdVv675b84PA5atmiJ5ngKrc/rw+P2oFKrMBhPzMHlkLua6wy6gLEfbviQH9N/pENcB2YOn4lKpcLldCEkgU6vQ6M9Ptbnw+OqYrunG6tSYTip95Pb5UbySWh1WrTHKw/XZqwkSbidsngwBhkRQrC7YDcLdy5kw9EN8k8BrRwnNDZtLBfFXIReo8cYJJ8zldeHyMlHU1SKKsiEQaXCGReF12Twu+AMJoP//fS4Pfi8PjRaDTq9fOEQQtR4LOB3wemNehbsWsAXO7/AiJFXLnmFhLCESmPdPjcWs8X/3ns9XrweL2qNGr3hRFxLVe9nVWNVbg8czgKfD01oMBqN+sTnxONFpVahN+g4WHyQjlEd6RPdBwEYDYYT2/V6cbndqFUqTCfFyDgcDiQhMOj1aLUn3k+ny4VKpSKojmOdTic+SUKv06E7fsGWJAnHcRO/+aR05tqMdblceH0+dFoter0eXnwRMXs29qFD4aOP5LFFRfDGG7g++0weq1Khv+EGmDYNER3tt/wEmUyB33uvF61GE/i9t9vB5yPo3ntR/forREfjXrQIT3R01WMBk9EY+L33eNCo1YHXiFqMtTscsrCtwftZm7Gnez/r8zmp6v0841hA53JBUBBSbCyO4+4h80liyOVy4fV60el08ntf8Tk5/n6eaawQAvvx817VNb82Y2tyf2iIe4n9uPvVaDT630+Px4Pb6UQjSRhPCg6vaqzX68XlcqFWqyu/n5IUMLahOWcKIfbr14++ffvy9ttvA/KHqmXLlkydOrVGAcsej4cOHTowYcIEv8mxvtuE8z8Ieu2Rtaw7uo62LdrWuSFpkaOIv/3vb7h9bl4Y+kKAO+ZcI7s8m//t/x/LDy7H6ZVvfqGGUEanjeaKNlcQpg7CUFiC1mrHYw4CjRptuR2f2YgztvEDJXfl7+KpX59CEhIP9XuIYa2GVRqTb8snz55HVFAU0ebo+u9UCPT5RehLbXhCqv9lXOgoRBISV6Rd0egVv5sNHg/07i03jPz3v+HyywNfP3gQZsyQs7VADo6+5x65gnJtrAwvvCA3MTUa4euvoUePhjuGpkCS5MB0g+GENaGp8PlOZDWFhckZTI3co0rh7HBOBEGD7C/96KOP+Oyzz9i9ezf33HMPNpuNW4/7lydNmsSTTz7pH//HH3+wcOFCDh48yOrVq7n88suRJInHHnusxtu80Mmx5vBn7p/EWGLqLH4Avtn9DW6fm/aR7ekW060BZ3j2iQuO486ed/LJlZ8wpdsUIk2RAXFCs9e9ybG8dDwWWfwAeIOMaG0ONLYaFu+rI1a3ldfXv44kJIYkD6lS/IAsSLvGdMXmsZFvy6/3frU2B/oyG96g6utC+YSPYmcxXaK7XDjiB+Su5AUFcpXb4cMrv966tZw2vngx9Ox5oiP7wIFyLZ3T1VOp4L//lcUPyOue6+IHZHeeRiOfj5ISOd38bCNJcvxKWZmc8deypezKUcTPBUmTOvCuv/568vPzefbZZ8nJyaF79+4sXbrUH8R85MiRgPgep9PJM888w8GDB7FYLIwaNYrPP/88IBDtTNu8kPFJPjZnbcbhcdSrUnORo4hl6XKAcXOP/akNFr2FqztczZXtrmTt0bUs2b2Y/SXp/Jizmh9zVtM7twtXJ42kZ3hHVBo1klqNrtSKL8jYKOmfQgje3fgu+fZ8Ys2x3NP7nirHWd1WgvRB9IrrRVJIEr8d/g1s1LqXWwUqtwd9YSmSRoPQVm+mzinPITEkkbaRbeu0n3OWL76Q/06YcHpLRp8+sGQJfP+9bBGq6MH18cdyE8/hw6u2Hq5bJ2ddgVyQ8Hyou+PzyY/4eDlupKREFiFWq2wVa2wBIoRsfXK5ZOETHy//PU+uXQp1Q+kFVgXnqwtsX+E+vt/3PYkhiXWu+Azw8ZaPWbJvCe1btOflES+fNwLoZLRWO7q8Qv4qO8BXBStZV7AVgfxVaWVO5G9tJtIzrAM6mwNnTAu8wQ0fQPlT+k+8s/EdNCoNL494mbYtqhYaB4sPkhqRyui00ahUKv7M/ZMVh1Zg0VtqL4Jq6PpyeB3k2fK4tPWlciHCC4WsLOjXT7YkrFolN+SsCW43/Oc/8MYbJyosX3wxPPtsYCf1Q4dgzBh5zJVXyu0pzofvV0mJXI8pIeHE8TidclmC0lL5/AQFNU6zUYdDfphMcm0fi+WsdCVXaBrOGReYwtnD4XGwKXMTBo2hXuKn2FHM0nS5/cjEzhPPP/EjBLpSK4bcQlQqNR3iuvJsl/v55KKZjE8cgVFj4JDtGE9uf42Xdn9Anq8cXalV/nXbgBwrO8ZHW+Tquzd1uala8eOVvPgkX0Ahyy7RXRjaaihWt5UCe0GV61WHxuZAX2bHG3T6X+RZ5VmkRaTRMqyBKhGfKyxYIIufiy6qufgBuXjcHXfI1p1775UtHuvWyfFD998PmZmyEJgyRRYL3bvLrq/z4fvl8cjHEREReDxGo1xwMDlZdkNV9GCr6MFVX1wueXuSJPelSk6Wa0Mp4kfhOIoAukDYkbeDzPJM4oPj67WdhXsW4va5adeiHT1iK8clOL1O9hbuJb0onRxrDuWucnxS8yh6dUYkCV1xGYb8IiSdVnZtHSfeFM09aTcxt//rjEsYgRoVK/M2cNuf01l8eCnqsho0C6whHp+HV9e9isvnomt0V67ucHW1YwvsBURbomkZekKIqFQqukR3YXDyYMpcZTUWQSqvD31JOZK6+l5fAMXOYoINwXSJ6VKvOLJzDkmSW0QA3FDHli+hobL7a9UquPr4+7pwodxYdNw4ufpyXBx88knzbMVRF6xWufxAdQ1FDQbZMpOcLB87yEX26trPzeOR13e75dIByclyzZpzPGVboeG5gK5eFy4F9gK25WwjKigKjbruv36KHcX+thLVWX8K7AUkhSTRPbY7Zp2Zcnc5B0sOsq9wH4dKDpFvy/fXjWlW+HzoC0swFJbgNRqQTkobP5lgnZl7297Ev3o/S7vg1th9Tt499g1/X/0ke3P+apCpfLb9Mw6VHCJYH8xD/R86rcgocZbQIbJDpU7wKpWKbrHdGJI8hDJXGYX2M9ek0ZZZ0dgdAcLvVCQhUegopFNUJ1oEtaj5QZ0PrFkjF1IMCYHRo+VlQsiWhtqSmAhvvy1nivXvL29j/35Z9MyZI1tGzgecTjlO6pSCgVWi18tCJTn5hKusuFgOnj6penG1+HyyFc1ul61NycmyAGrmHckVmg5FEp/nSEJiS/YWylxltGvRrl7bqrD+tG3Rlp6xPascY/PY6JfYj55xPZGERLmrnBJnCSXOEnKsOeRYcyh0FOLwOBAITFoTZr0Zi85S6SbeEAgh8AkfPsmHTqOrUkyoPF70haXoysrxmoNOG/hbQVpwCm/2epql2av4d/rXpNuP8eiKJ7ms9WVM6jbJX127tmzO2sySfUsA+Hu/v9PCVL3IKHOVEWIIISUspcrXK0SQQLAqYxVAtaJF7XShLy3HazKe1u2Sbc0mPjie9pHta3hE5xH//a/896qrTlhnKgJrVaq63Wi7dZNbaPz8s+xeu+UW6Ny54ebc1NjtsnurNrE9Wq1sMQoOlsVPUZHsFtTpqu7nJknyOK9XFqcREbK16XxwHyo0KooAOs85XHKY3fm7SQyue9YXyG6PCuvPDZ2qzvxyep3oNXriLLIZW61SE2oMJdQYSjLJdKMbXslLqbOUEmcJxY5isqxZFNgLyLZm4/a5UaHCpDNh0Vsw68zoNLoAEeOVvPJDeP3PfZLP/1wSEqiAkwxMGrUGrVqL3WOnVVirgBgotduDPr8Yrc2Bx2L2p7nXBLVKzaj4oVwc2ZNP9s9nWd46fjr4E+uPrWdyt8mMaD2iVi6iYkcxb/7xJgCj00bTN6Hvacfn2/LpGN3xtJYYlUpF99juCCFYdWQVqKgsqiQJfXEZ+CREUPVZTQ6vA6/PS9forph054l7pqYUFcFSOfaNG288sdztlt1aFd2w64JKBZdeKj/OJxwOWfjUtaO6Viuva7GcEEKlpXIMT4UQcjhkK5OS2aVQBxQBdB7j8rrYnLUZlUqFWV+/LKVFuxfJ1p+ItvSMq9r6U+woJtIUedrMI61aS4ugFgE3bZfX5bcSFTmKyCrPothRTKGjEK/kRYXKL2I0KvmvVq1Fr9ETYgjBqDVi0pkwao0YtUZ0ah16jR6dRodOrUOn0aFRadias5UduTtICknCrDejdrowFJSgcbjkjKc6XjjD9CE83OlORoX3441jX3HYdox3Nr7Dzwd/5p7e99AqvNUZtyEJiTf/eJNSVynJocnc2v30dasqWmKkRaSdcdsqlYoecXK81qojxy1BJ4kgrdUuC8Cg04ua7PJs2rZoe2FlfVXwzTey2OncubKFJihItgJVZBopyK5Bh0N2ZdW36KFGI1t2KoRQRQq9EPL5TkyUrUVKcLNCLVEE0HnM7vzdZJRk1LtvVImzhB8OyFVtT5f5Ve4up2tsV7Tq2n2sDFoDMZYYYiwn4h7sHjslzhIcHkeAkDn5r1atrVUW2iWtLsGgMbA5ezNJ6nAiy7yoPD48wQ1jLu8Q1ZEPQh5ngWMjX+z5ij2Fe3jop4cYkzaGG7vcSJCumiBQ4Nu937I1Zyt6jZ5HL34Uveb07pR8Wz4xwTE1rudUIYIEgtUZq1GhIsIUIdf8KS5D0mpPa/0qdhZj1pvpGtv1wgp8BvlGW1Xws88nWyEqrBGZmbLFQ7FAyELFbJaFS0OhVstCp0IIeb3y/0pws0IdUT455yklzhK2/H97dx7fVJn1AfyXpNm67xuUtpR936QgqKgooKIgw7Api7gMI+NSV1Q2xxHFUdERxVFwFxFFXGBwQcFRUBREZF5BQHa6UZqlSe7Ndt8/Hm/atGmbm9wsbc53Pv0Y0uTmaemQ0/Oc55yKPUjVpUKtCu43sA8OsOxP1/SuGJw32OdjHC4HVEoVOiR1COq1RPHq+BYDhkBoVBpcUHgB9BY7ft3/FYy6VCSm58p2fUEdhzhOgcnZF2NE8UVYvXcNa6j420f45uQ3uHHgjRhRMKJJ0Hb43GG8se8NAMCNA2/0OtHl83UEAWa7GcMLhkv6u1UoFJ7s3X+P/xcQBORaFFDanS32/BELn4d1GIbM+Ey/X6/d+Okn4MABFtxMnFh/v93O6n40GpblMBrZiaekpMitVWQyscCguZNXoeR2s+9Nbm5osjIKBQt8CAlSjP0qFxsEQcBP5T+hxloT9FwoA2fA5kOtZ39quVqk6dO8sjhRRxAQV2vEUEUHDCkYhnMqO2psMkzsbsCl10JttiJHkYT7RtyHJRctQV5iHs7ZzmH5juVYvG0xTptOex5vdVjxxI4n4HQ7MbzjcIwpGdPqaxg4A1J1qc0WP7dEDIJGdhoJ87kK2KrOtNrzp6KuAnkJeeiZ1VPy67ULYvHzlVd617M4HCwoUirZG31GRn3H40iy2dh6nE5WHxNudXX1tTuERDEKgNqhU6ZT+F/1/5CflB90o8KNBzaCd/Hokt4FQ/KGNPs4I2dEcWpxq1s3EeN2A9XVQEUFlPEJ6FN0HkYWjgTn4lBRVyHbywhxcayZoqkOEAQMyhuEf437F6b3mQ61Uo29lXvxty1/w1u/vAXeyePfu/+N8rpyZOozMf+8+X79fZ21nUXX9K5I0QVWXKpQKDA4uz8uSOiFWrsJBqel2cfyLh52lx39cmOw8Blgb+YbN7LbDYufARboNMywJCayN/46+XpCSeZysaAnM5PNKrNa/Zs9Jhenk/1/LS0tJONhCJETbYG1Mw6XA7vLd8PpdgZ8FFtk5IzYdGgTgOZPfgGsGzEUQEFyQVCvF1I1NUBVFdueUKuhANAjswc0Sg12nNqBU6ZT6JDcAQoEX7/hjNdDXWeFKjEersR4aFQaTO0zFRcVXoQX97yIPeV7sO5/6/DZkc9Qy9VCqVCibHgZkrStb51wTg4qhQol6RK6EPugMBrRN6EI7i5K/HDmR0ABpGibBlRnzGdQklYSULapXfj4YxZEFBezERgiQWh69F3sdmw212+PhZvZzIKwlBS2HoeDDW5NTQ1PQFJXx15LytR7QiKEQvR25rea33D43GFZgpGNB//I/qR1wZD8lrM/qbpU5CbKV08jK5uNHaFNSGhyIqVzemdcXHQxEjQJOGE84Zn3FRSVEoJCwUZkNGjglpeUh8UXLsb9I+5Hpj4TtVwtAGByr8nok+1f75cqSxU6JHcIrqO3zQbU1ECZmIT+eQMwJH8Iam21MPJGr4cZOSP0aj365fYLqoFmmyYOPp02zbu42eFgNTaNgxy9ngVBkcgC2Wzs5zszkwU7CgW7nZzMaoJCzW5nW29paVQITtoEygC1I2bejB/Lf0SSJinorSgTb/Jkf1qb+WXgDeib3Tc6t0gEoX4eUDMTpzskd8DFxRfjm+Pf4GjtURSlFgV90skZr4P6j+PlDQelKhQKnF9wPgbmDvQUl0/tPdWva7oFN2xOG3pm9gw8IHG7WUbA5QK0WigBDMgbAAD44cwPUECBZG0y3IIb1bZqDM0fGnQdWZt18CCwZw97U5882ftzDgcLfnydQEpNZQFHOI/Fu1z1x84bNh2Mi2PdkB0OFpSFsi7HYmEBF7UCIG0EZYDakX2V+1BVVyVLJuaDAx+Ac3IoSSvBefnnNfs4t+CGy+1q9eRSxJjNrG9IK//wZydkY1TxKOQn5+N3w+9sWy8YSiXccapmB6Xq1XpM7zsdswfM9juYOWc7h3R9enDbUSYTO63U4KSSUqHEgLwBOC//PNTYamDmzai0VCInIQe9snoF/lptnVj8fNllLIhoyG5nGUVfvxiIIx3kGurpD7OZBV6+mg6KQ0fd7tAVRdts7Ov2Z+QFIVGCAqB2otxcjn2V+5CTmBN09kJK9sfIG5GsTY7O7S+nk9X+qNV+HcdN16fj4qKL0TmtM44Zj3maDQbKpddBZbVBXWcL6jqiGlsNumd0D7yppd3Osj86XZPvhxgEDckfgrPWs+CcHPrn9ke8JgLHqH3hOODYsfC9Hs8D773Hbjc3+LSZjCKA+sZ94dgKs9lYpkfc+vIlMTF0RdFi08P09Ja/J4REGQqA2gGX24Xd5bthc9iQpksL+nobD2wE5+TQOa0zhua3PI7BYDOgY3JHvwp4w85orG/I5qckbRIuKrwIPTJ64ITpBGzOIIIXhQJujQZqgxkKR3BvOlaHFbo4XeBNLcWtQJ5vtjeMUqHEwLyBGNJhCLpldENxausdrMOirg644gpg5Mj6E1mh9umnbBBnbi4wapT358QGiC0VOYfrWLzbzYKPrKzW522lp7MgyWTyb7iov8StvkBHXhASIRQAtQNHao/gUM0hFKQEX/jslf3p3XL2RxAE2N326DwhxPPsDV+vl1yQqVfrMaLTCPTN7otycznq7IH/Fu/SaaC02xFnbv6ouT8qLZXolNIp8EybOEupla1ApUKJwfmDcVHRRdFR+CwIwJ13snocQQDuuQc4fDj0rytuf02Z0rTOp2EDxJaE41i8yVR/6qs1oSiKFrfVMjKoIzNpcygAauOsDit+PP0jtCqt15DPQH148EPYnDYUpxajtENpi481281I1CRG3/aXmO2w2wMuyNTGaTG8YDgG5Q9CtaUaRs7Y+pN8USjg0mqgNtVBaXcEdAmX2wW7y47uGd0D6+vkdLKtL5XK77lMUTPuYuVKYPNmtu4+fdgWzs03s/+GysmTwNdsZhqm+ihQb9gAsSXisXiFgv0sys2fra/G4uJYPZBWK09gZrGwQC8aul8TIlGU/CtHArW/aj9Om08Hdyz6DybehE9++wQAMK1P831/RAbOgLzEPKTqUoN+bVlZLGz7IsgTL3HKOAzJH4KhHYfCwBsC7hrt1mqgtDsDzgKdtZ5FVkIWClMLA3o+DIbQnwAKhe3bgccfZ7cfeQR44w1WjHzwILBgQegKjNetY/8dORLo5KO4v3EDxJbo9exYuNxZoIZbX1KDfK1WnqJol4sF1xkZNIiUtEkUALVhZ61nsbdiL7Lis2TZrvjo4Ed+Z38Aln3qnNY56G7TsnK7WeGzUhn8FGr8URycOwAjO40E5+RQaakM6DpOvZZlgXjpmYBarhY9MnoEluH7o+cP4uUZ+Bo2J04Af/0r+/ucNg2YMYMFP88/z/5u33uvPlCRk8tVP/i0cednoD7oktLkMC2NZYxs8hTDA/BueBiIxET2/QymKFpcQ1sLrAn5AwVAbdj/qv4HE29CRnxG0Ncy82Z8/NvHAFo/+QUAFrsF8Zr46Nv+MpnYP8wy/qOsUCjQM6snLiy8EFAAp82nJTdMFDRqKJxuxBnNkp5n5tk2Y0DFz2Iw6Ha3XiAbTWw24MYbWeZqwACW/RF/HocPB+69l91+8EHgf/+T97W3bwfKy9lx7jE+5rI5HCywlhIAyX0sXpz1JWXry5e0tMCLoh0O9ndCTQ9JG0YBUBtWba1GkkaevXcptT8Ay0pkx2fLEnzJxm5nb/j+1GcEoCS9BKMKR0Gn1uGU6ZTk5zvj2aBUlY33+zlV1ioUpRYFNoXdzx5ITRw8yIrII0EQgPvuY4FNRgbw7383Dd5uvRW45BK2fXPLLezrlItY/Dxpku+gsaUGiC2R61i8uPWVkRF8w0GxKDolRXpRtDjyIhLT5gmRCQVAbZRbcKPOXifL8NGG2Z8pvaf4VQBrsVtQkl4SPcWyAHuzt9lC+o9yQUoBLux0IfRxepTXlUt6rjgoNc5Y51cmwOFywC240S2jm/RtxhZ6/rRo82YWXFx1FVAh35BYv736KvD++2zNL7zAOhs3plQCzzwD5OcDR48Cd98tT2aluhr47DN2u7nePw5H8w0QWyLXsXjx1Fda8O0uANR3ipZSFM3zLAtG2R/SxkXRuxeRgnNycLgcUCuDr3P56DdW+1OUWoRhHYf59dqaOA3yEvOCfm3ZWK1+HfOWQ4fkDhheMBwAy8JJwQalWqCytl58Wm2tRk5CjvT2BuIpOI6THgyuWsX++3//B1xzTXiOnIu+/x5YsoTdfvBBYMSI5h+bns7WGhcHfPIJC5yC9d57rB5m4ECgZ0/fjxGEwJv9icfiA81YcZz0U1/+kFoUbbHU1zUR0oZRANRGcU4ODrcDalVwAZCJN9XX/vSe6ldGp9ZWi0x9ZvTMiGo47ytME7iL04oxrOMwcA7OM9TUL+KgVIO5xboLQRBg4k3omdlTepbPz54/TezbB+zezd5ki4qAU6dYEPTDD9KuE4iKCrad5XSy17z55tafM3gw8NBD7PbSpcDevYG/viB4Dz71xZ8GiC0Ra2aUSunH4t1uFuTLsfXlS2IiC4JaK4q22VjgQ00PSTtAAVAbxTk52F32oLfAPjjwAawOq9/ZH4D1/ylJL4mORnkA+4260XyrcOiW0Q1DOwyFkTPCzPv/W70zXoc4qw1xLWSBjLwRybpkFKdJ7Mbscknu+eMhZlGuugr4+GOWCTEYWC+c//xH2rWksNtZwFNdzTIv//yn/1srN97IukQ7HCyAMhgCW8MPPwC//84yZtdc0/w6/WmA2JL4+MCOxYtbX6GctZWayo7VN1cULY68yMgI2y8ahIQSBUBtFOfk4HK7EKcMvPuqgTN4+v5M7zPdr+yPw+WASqmSpe+QLMQmf37O+5KTQqFA75zeGJQ3CFXWKlgdfjbnEwelGszN1oNUW6tRklaCNL3EWo/a2sB6/pw7Vz9mYvZstsW0fj0bBMpxwE03ybPN5MvixSzzlJICvPyytG07hQJ48kmgsJBlrO64I7AxD2L25+qrm//eOZ3yFNhLPRbPcfWnvkL5M96wKNroo/Gn1crqn6jpIWknKABqozhn8FOd3//1ffAuHl3Su/h18gtgp7/S9enIScwJ+vVlYTDU/8McAeL8rP45/XGm7ozffy/ioNQ4S9M3Qd7JQ6lQomt6V2mLCabnzzvvsOLWPn2AIUPYfXo9C0hmzGC//T/4ILBsmbwNCNetA15/na33X/9iW29SJScDL77Ialk+/5zdlsJkYhkvoPntL4AFQHIU2Es5Fi9ufWVmhmbrqzGVihVF63TetUpuN/v5SE+nkRek3aAAqI0KNgCqsdXgP4fZtsaMvjP8PmVk4AzonNZZltNnQeM4lrmIcJM/lVKFIR2GoFdmL5wyn4LD7cfIiz8GpWpqTdDUGKCpMUB9zgj1OSNM5cdQ5E5GB1c8y+gYDOzDaGQfJlN9vyOzmWV86upY8ONySS9OdbmA115jt2+4wft7GRfHujHffTf783PPsSyLI7CxHl5+/pl1dAaAu+4CLr008Gv17cvqgAAWpO3a5f9zN25kP0vdurG6Il8CaYDYEvFYvKWV7uAmE3tsKLe+GhOLogWhvii6ro6tg7I/pB2hUL6NqrPXQaUIPB3+3v+9B7vLjh6ZPTAod5Bfz3G6nVAoFOiQ5ONocrgJAgsOHI6o6ESrUWlQ2rEUdpcdh2sPozi1uNW/H5dOA5WNg7pW/E1bYP+rLUfPTiWIq6iqf7AgsMCkuf+KjwmkOHXrVrZ9lJrKtoAaUyjYQNK8PNaE8L33WL3Ov/8d+Pe+pobV7/A8cPnlwO23B3adhq67jgU+GzYA8+axie6ZfvRPEnv/TJvWfCAdSAPElojH4k+cYAGor62tcG19+SIWRZ85w/7sdrPsTwj6axESKfTT3EaZeXPAJ8CqLdX49MinAKRlf4ycEam61Ojo/izO+4qi30jFKfKFKYU4ZjgGt9BKLYpCAVe8Hs6k+D8+EnBW7YA+Kw8dCnqzWhHxIz295f+KtwN5o3zlFfbf6dNb3maZOpXVAen1rGPypElAVVXzj2+O08kClDNngOJi1tNHjjdWhQJ47DGgSxd2qmz+/NZ77uzfz06/qdXAn/7U/OMCbYDYkpaOxTfc+opUs8HUVPb6dXVsnRHaZiYkVCgAaqPMdnPA21Dr/rcOTrcTfbP7on9Of7+fZ+ANKEwphF4dhlqElrhc9fO+oqweIUGTgBEFI5CXlIfjxuOSR2bU2mpRnFaMRG2YslqHD7PJ5woFMHNm64+/5BKWAcrIYMHD1VdL7xW0bBnw7bfsjX31ara1IpeEBJaZ0uuB//4XePbZlh8vzv0aM4YFkM0JtAFiS8Rp8b6OxZvN4d/6akwsis7Orp9qT0g7QgFQG+R0O8E5uIAyQBV1Fdh6dCsAlv3xl1tww+V2oVOKj+nY4RaCeV9yStWnYmSnkUjXp+O06bTfz+Oc7O+0KKUodItrTDzZddllQIGfDRcHDAA++ogVLJ88CUyYAPz4o3/P/eij+maLTz8NdO8ubb3+6N6dZYIAdkLs6699P85mY9tlgO/Bpw253YE3QGyJr2PxHMeCokhsfTWmUgG5ueEpwCYkzCQHQIsXL8bx48dDsRbiJ87JgXfz0CilZ4De2f8OXIILA3MHoldWL7+fZ+SNSNGmRH77S5z3pddHdT1CZnwmRnQaAb1ajzPmM349p8ZWg7ykvPCdsDObgXffZbfnzJH23KIi4MMPWTBUWwtMmVI/RqI5Bw6wYmeATXq/6iqpK/bfn/7EghpBYFthvsZ6/Oc/rKi8Y0fggguav5ZYoxOq3jcNj8U3bHhIc7YICSnJ7yAffvghSkpKcOmll+Ltt98GH6mhiTGMc3JwupySM0CnTKew7fg2ANKyPwBgsBnQIbkDkrQRrrmprWW/IbeB30jzk/JxfsH5UCqVrY7McAtu8E4eXdO7hm++2vvvs1qqkhJg5Ejpz8/MZL2CxMGkc+cCb7zh+7FGI/u81cqCjfvuC27t/nj4YaBXLxYw//WvTTsci71/pk5tOZgWGyBKbSzpr4bH4k0mVtcm16wvQkizJP9Lu3fvXvzwww/o3bs3br/9duTm5mLevHn4IcB2+StXrkRRURF0Oh1KS0uxq5XjqytWrED37t2h1+tRUFCAO++8E1yD+TVLliyBQqHw+ujRo0dAa4tWYhdoqXPA3tn/DtyCG0Pzh6JbRje/nycIAuxuO4pSiySuVGZWKwuAonTry5fC1EIM7zgcvJPHOe5cs4+r5WqRqk9Fx+SO4VmYINQXP8+eHXg2LT6eXWfaNJa9uP9+YPly7/42bjdw223AsWMs2/L88+Gp3dLrWU+gxEQ2Z2z58vrP/f47sHMnq2v5859bvo7YADGU21HJyazGSKlk3ZgjvfVFSAwI6F+9gQMH4tlnn8WZM2ewevVqnDp1CiNGjEC/fv3wzDPPwOiri6gP69atQ1lZGRYvXow9e/agf//+GDNmDKqaOVny9ttv4/7778fixYvx66+/YvXq1Vi3bh0eeOABr8f17t0b5eXlno9vvvkmkC8zanFODm7BLWkUxXHDcfz3xH8BANP7tlLv0IjZbkaSJimy219uN/tNPozzvuTSNaMrzuswFGbODBNv8vkYA2dAl7Qu4Ssw/+YbVryckABMnhzcteLigCeeAMrK2J+feYZtdYm9glasAL74ggURL7/ccrGx3Dp3ZqM1AGDlStYoEWANGAHg4ot9T5xvSK4GiC0RGxDm5NDWFyFhElSuXRAEOBwO2O12CIKAtLQ0PPfccygoKMA68R+YFjz11FO46aabMGfOHPTq1QurVq1CfHw81qxZ4/PxO3bswIgRIzB9+nQUFRXh8ssvx7Rp05pkjeLi4pCbm+v5yPSnF0gbEkgTxLf3vw0BAs4vOB+d0zpLeq6BMyA3MRepulTJryubCM37kkuf7N4YlD8IZ61nm4zMsDqs0Kl1KEwtDN+CxOLnyZPl+Z4qFCzoWb6cZTHWrWN1RR9+yAqRAXb6q2/f4F9LqvHjWYNHgDVxPHasvvZp6tSWnyt3A8SWJCTQ1hchYRRQALR7927Mnz8feXl5uPPOOzFw4ED8+uuv2L59Ow4dOoR//OMfuO2221q8ht1ux+7duzF69Oj6xSiVGD16NHbu3OnzOeeffz52797tCXh+//13bN68GVdccYXX4w4dOoT8/Hx07twZM2bMwIkTJ1pcC8/zMJlMXh/RTGoAdKT2CHae2gkFFJjeR1r2B2Bv0J3TOvvdL0h24rwvrbbNbg0oFAoMyB2A/nlNR2bUWGvQMbEjMuPDFKifOlVfsDx7trzXnjEDWLOGZXu++orV3oiv09pWUyg99BAr2DYY2LDTqipWd3PZZS0/T+4GiISQqCE5AOrbty+GDRuGo0ePYvXq1Th58iQee+wxdOnSxfOYadOmobq65aLPs2fPwuVyISfH+8RLTk4OKnyd2AAwffp0PPzwwxg5ciTUajVKSkowatQory2w0tJSvPrqq9iyZQteeOEFHD16FBdccAHMvpqN/WHZsmVISUnxfBT4exw4Qsy8WdIQ1Ld/YcWeFxZeKPkYu8VuQYImIbLbXwYDKxCNhq0Bmw0oLw/oqSqlCkPyhqB3Vm+cMp+C3WWHS3DBKThRkl4SvgDzjTfYVuLIkUBXifPG/HHZZaw4WtzqOu88NvA0krRadvw+NZUF0wDLfrUW2ISiASIhJCpIDoD+/Oc/49ixY9i0aRMmTJgAlY/fyDMzM+EOZCJzK7Zt24ZHH30Uzz//PPbs2YMNGzZg06ZN+Pvf/+55zLhx4zB58mT069cPY8aMwebNm2EwGPCumPL2YcGCBTAajZ6PkydPyr52OZnsJr+bIB6sOYgfzvwApUKJqb1bSff7UMvVIis+CxnxGZKfK4tomPflcrFeMnfeybIIQ4YAV17Jmuj5O9H7D2qVGqUdS9E1vStOmE6gylKFDH0GOiSHabwIxwFvvcVuSz36LsWgQcCmTcCiRSwjFA0ZlIICVo8kamnwqSgUDRAJIVFB8q81CxculOWFMzMzoVKpUFlZ6XV/ZWUlcnN9ZxsWLlyI66+/HjfeeCMAlo2yWCy4+eab8eCDD0Lp4yRLamoqunXrhsMtdKvVarXQhqLJWYjU8XV+nwB7ax97s7u46OKA3mQtdguGdRwWvqPZDQkCC34iMe9LEFin4w0bWB1Lo59T7N3LPh5+mPWcmTmTjWDwgy5OhxEFI+BwO3D43GEMyBkAbVyYfv4++oidpOvQAWiw/RwSnToBt9wS2teQ6rLLWCG2y+Xf31eoGiASQiJO8rvapEmT8Pjjjze5f/ny5Zgs4TSJRqPB4MGDsXXrVs99brcbW7duxfDhw30+x2q1NglyxAyUIPgeOVBXV4cjR44gLy/P77VFM7vLDt7F+5UB+l/V/7C3ci9UChWm9J4i+bU4JwdtnDZy218mE9v+Cmfh8/HjLEswahQwdiwbq1BZybZOrruOBUR797Ip5gUFrDB79WrgoovYlsrHH/s1KT1eE48RBSMwIHcAitKKQvs1iRoefZ85M3a3dcaN868JY6gbIBJCIkryv4Bff/01lixZ0uT+cePG4UnxtIefysrKMGvWLAwZMgRDhw7FihUrYLFYMOeP1PzMmTPRoUMHLFu2DAAwfvx4PPXUUxg4cCBKS0tx+PBhLFy4EOPHj/cEQnfffTfGjx+PwsJCnDlzBosXL4ZKpcI0f9LdbYDYA0ivbfm4tCAIeOsXlv25rPNlAQUxtbZaZOgzkJ2QHdBag2KxsMAjHPUX586xzMgHH3iPdNDpWJbk2mvZcemGb4Tz57MC323bgNdfZxPVd+xgH9nZ7HTRdde1eMQ6RZeCCwsvDN3X1dhPP7HBn1qtf9s/sS7UDRAJIREl+Z2lrq4OGh+/EanVasmnp6ZMmYLq6mosWrQIFRUVGDBgALZs2eIpjD5x4oRXxuehhx6CQqHAQw89hNOnTyMrKwvjx4/HP/7xD89jTp06hWnTpqGmpgZZWVkYOXIkvvvuO2RlZUn9UqOSGAC1lgHaV7kP+6v3I04Zh8m9AuvzYrKb0D+3v6R+Q7LgODa6QBBCV/hsswGffsoyOtu313cJViiAESNY0HPFFS1nn5RK1gX5kkuA06eBN98E1q5lJ4yefRZ47jng0ktZtmXUqMiP7hCzP1dfzU5AkZY5nWzrtY2ePCSEtEwhNLd31IyhQ4fiqquuwqJFi7zuX7JkCT7++GPs3r1b1gVGgslkQkpKCoxGI5LlnFQtgxPGE3jvf++hJL2k2bocQRBw3xf34UDNAVzV9SrcPPhmya9jd9lx0nQSk3pOQkFKGE/F2e3AmTOs67Pck7CdTtYAcMMGYMsWlmUS9e0LTJzIjkg3U4PmF4eDXfv111k2SNSpE8sITZ0ameCjupqdxnI4gM2bgf79w7+Gtqa2FsjPp948hLQhUt6/AyqCvvbaa3HkyBFccsklAICtW7di7dq1WL9+fWArJn7jnBwECC0WJe8u340DNQegUWnwp15/Cuh1DJwB6fr08A3mBFiAUlnJJmPL+aZTV8c6FX/4IQsERJ06saBn4kT5joOr1azx3vjxrNPy668D770HnDgBPPoo60p8xRUsKzR0aPhOF731Fgt+Bg1qPfgRBFb8G22ZD6s1fEfSw9kAkRASEZL/JRk/fjw2btyIRx99FO+99x70ej369euHL774AhdddFEo1kgaaK0JoiAInr4/V3a9Eun6wMYOGDgDSjuW+n3cPmhuN9s6MhpZ5kfOwGDRovrRB2lpbAto4kR2nD2UAUiXLuyU2IIFrMbo9ddZAfXGjeyjRw/g5ptZg8BQrsPhqB9S6s/Rd7OZPSeatslcLrY1arfLnxn0RWyASPU/hLRbAf0qdeWVV+LKK6+Uey3EDzaHrcWGed+f/h6Haw9DF6fDtT2uDeg1nG4nlAolOiSFqTeNILDMzLlzQEqKvLUyZ86wLS+Azai65prwv6np9cCUKexj3z4WCH3wAXDgAJufdfQom44eqiDo009ZTVVmJutf1BqnkxVKi0XA0YDnWT0Yz7P1hToL5HRSATQh7VyEqzKJVGZ7812g3YLbc/JrfLfxSNGlBPQaRs6IFF1K+I6/nzvHAqCkJPm3XV56if02P3w469cT6Te0fv3YNtiePfXDQ//1L+Cxx7wnqMtJnPs1Y0brPW3EoEenY8FGtOB59vORnOxduxUqdntkm28SQkJOcgDkcrnwz3/+E0OHDkVubi7S09O9PkhomXgTNErfv5XvOLkDx43HEa+Ox4TuEwJ+jVquFkWpReGZTG40srqfhAT5gxODob7rsTiTKlqkpLDhoQ8/zP783HOhCYJ+/RXYuZMFltdd1/rjeZ79XSQn+9XPKCwEgQUi8fHs+yYIbEsslNxuFgQSQtotyQHQ0qVL8dRTT2HKlCkwGo0oKyvDtddeC6VS6bM/EJGPIAios9f5rMtxuV14ez+r/ZnQfQKStIE1D3S5XXALbhQkh+HkV10d25rRakPTbfe111i2oGdP1scnGs2dC4ijXJ57jk1MlzMIErM/Y8eyE02tEbtu63Rsm0lsDxBJHMd+PnQ6FgQlJLCC6FChBoiExATJAdBbb72Fl156CXfddRfi4uIwbdo0vPzyy1i0aBG+++67UKyR/IF38bC77FCrmmZKvj7+NU6ZTiFJk4Sru18d8GuY7CakaFOQlxTiztk2Gwt+FApWIxOK669Zw27PmxfdWxk33FAfBK1cyU6LyREEGY3A+++z2/4UP4uFvzpdfcARDdtgPF/fj0epZIXsTmfotgypASIhMUFyAFRRUYG+ffsCABITE2E0GgEAV111FTZt2iTv6ogXzsnB4XI0yQA53U6s/d9aAMDEHhMRrw68eWCtrRYdkzsiURPC2Vs8z4KfUM74Wr+eTf3u0IGd+op2N9wAPPIIu/388/IEQe++ywLBHj2AYcNafzzPs2BUo2EBY1JSdARAjRtiJiSwP4cqC+R0suAv2toAEEJkJTkA6tixI8rLywEAJSUl+OyzzwAAP/zwQ5saKNoWiV2gGw9C/fLol6ioq0CKNgVXdg38dJ4gCHC4HShMLQx2qc1zOFjwY7WyOpNQcLmAF19kt2+5pe38Jj9nDiB2NX/+eXY70CDI7a7f/pozx78MmN3O/k7Ex+r1LOPidge2BjnY7fVZKZFKxbJAPB+aLJDTGboO5ISQqCE5AJo4caJngOnf/vY3LFy4EF27dsXMmTNxww03yL5AUo9zcnC6nV6nwBwuB97937sAgEk9JwVVuGy2m5GkSQrd9pfLxXr9mM3y9/ppaNMm4Ngx9ibZ1mZezZ5dHwS98ALLCgXyJr9tG/seJCezsR6tEY+WNww0xK2wSGaBxOPvjYPYxEQWoHEt98WSjBogEhIzJDfTeOyxxzy3p0yZgsLCQuzYsQNdu3bF+PHjZV0c8SY2QWzYB+jz3z9HlbUK6bp0jOsyLqjrGzgDOiZ3RIo2sOPzLRIEtiVVW8tO8oQq+BEElj0BWOajLf4mP3s2+++DDwKrVrGvaeFCad8zce7XlCn+fQ94vj7gESmVbBusujo0dVr+aG6bNC6OBdHl5fKujRogEhIzJGWAHA4HbrjhBhw9etRz37Bhw1BWVkbBTxiIYzBEvJPHu//Hsj+Te0+GNi64LUirw4rOaZ1bbLQYEEEAampYAJScHNraim++AX75hb2Z+1P4G61mz2Z1QADbznv4Yf8zQUePAl99xW7PmuXfc3jee/tLJAZPoSo4bomvrFRDSUksYJMzC0QNEAmJGZICILVajffFUyUk7KwOq9cMsC1HtuCc7Rwy4zNxeefLg7q2xW5BgiYhNM0PjUa29RUfH/oOvmL2Z9o0oK33pZo1ix2LB4B//9v/IOj119njLrkEKC5u/fHisW9fmZRIboP5yko1pNGwbKKcxdDUAJGQmCG5BmjChAnYuHFjCJZCWtOwCSLn5PDe/70HAJjSe4rPo/FS1HK1yIrPQka8zPOfzObQ9vpp6JdfgK+/Zm/mt9wS2tcKl5kzvYOgpUtbDoKs1vq5Z+JWWmtaCjTi4tipK7td0rJl0bgo25fkZJatkWt91ACRkJgh+dfxrl274uGHH8a3336LwYMHIyEhwevzt912m2yLI97MvNkT6Pzn8H9g5I3ITcjFpcWXBn1ti92CYR2HtThlXjKrlQU/zWUX5CZmf66+GigIQyPHcJk5kwUB99/PRnsIArBkie/A4IMPWMatsND/5o88D+TkND+DLSGBbWGGk9vNvr7WghGdjgVBtbXBFy5TA0RCYorkAGj16tVITU3F7t27sXv3bq/PKRQKCoBCxOV2weKweHoA7SnfAwC4psc1zc4G81edvQ56tV7e0188zwpUXa7QHXdv6Ngx4JNP2O1580L/euF2/fUsILjvPuDll1kQtHSpdxAkCPXFz7Nm+TdUVgw0WiqU1ulYlkUsEA4Hnq9vxtialBQWAAU7JNXhoPofQmKI5H8tGhZAk/DhXTwcLoenyWGVpQoAUJgSfM+eSkslumd0R05CTtDXAsDeSMrL2ZtYaqo812zNiy+yN/OLLwZ69w7Pa4abOMvrvvuA1avZ7YZB0K5dbPaXOH3eH/4EGhpN/ZHzcAZAmZn+BXF6PQuy6+qCC7bFE2fUAJGQmEDT4NsIzsnB7mZNEN2CG9XWagAIOmjhnay4tWdWT3lOfzkcbLipxcJ+Mw+Hs2dZ12Mg+oaeyu2664Dly9nt1auBxYvra4LE7M+11/ofeIqnv1oLNJKSwjccVRCadn9uiULBvl6XK7imjdQAkZCYIjkD1FqzwzXi/CUiq4ZjMGpttXC6nVApVEjXB3fSqdJSiU7JneQZfmo2s9NeNltoGx02tmYNy04MGAAMHx6e14ykGTPYf++9tz4TNG8e8J//sNv+Fj+LgZM/b/o6HQuSxDqZUJKy/SVqOCQ1kPEq1ACRkJgjOQCqra31+rPD4cD+/fthMBhwySWXyLYw4k3sAq1SqlBpqQQAZMZnQqUM/M3I6XaCc3Lond07qOvA6QTOnWOZmLg41oE5XMGPxcKmvgMs+xMrx5dnzGBf6z33sCBo61b29zBsGNCrl3/XaDhlvTU6Xf1w1FBnScStUyn1PEola3tw4gQLZqT+HFADREJijuQA6IMPPmhyn9vtxrx581BSUiLLokhTnJODAuwfdbH+JzshO6hrVlmqkJuYi6LUosAvYrGwTsF1dew373D/Bv3WW4DBwPrdjB0b3teOtOnT64OgY8fYff5mfwAWaGRk+JfREYejVlaGPgByuVg2R6qEBFYPZLNJXyM1QCQk5shSA6RUKlFWVoann35ajssRH8QxGIA8AZBbcMPEm9A3p29gHaRdLpbxOXmSZRLS0sIf/NjtrDcOwLaAYrF4ddo04J//ZAFKQYH/QaDUOhugvpVBKIejipmYQHrxqFQsC8Rx0jtXUwNEQmKObG15jxw5AqfTKdflSCN19jqoFOwNXo4AqMZag4z4DHRO6yz9yTYbC36MRvZbd6gbHDbnww/ZabPsbGDSpMisIRiCwN6sdbrg3ninTmX1T6mp/mcwxDobKf2ZGm6Dhaqvk3jtQIPpxES2Ro6TtkZqgEhIzJEcAJWVlXn9WRAElJeXY9OmTZjl79whIlnDJoiyBEC2GlxYeCESNRIKRt1uFvRUV7Mtg9RU/44ph4LbzaalA8CNN7bNNy+jkWUt6urY9lIwevSQ9vhA6mxUKhZg1NSELgCy24GsrMCfr1azr6uiwv81ioXdtP1FSEyRHAD99NNPXn9WKpXIysrCk08+2eoJMRI4k93kaYJYZWUBUKBH4A2cAcnaZHTN6Or/k3ieZX0MBhZsBHLSRk5btwIHD7J1iP1x2hKDgWVgxDdrtzu8wWSgdTbx8SwADqTQ2J81qVTBB7NJSawoX8xytUZsgEgnwAiJKZIDoK/EKdMkbBwuBzgHB7WK9QAKNgNUbanGwLyB/h2hFwTAZGJvemLPmGiotRHHXlx/ffj6DcnFaGRvzPn57L91dayYPNgskL8C2f4SiTPD7Hb5tz7FmWTBBkBaLfuZOHvW/wCIGiASEnMk/8p59OhRHDp0qMn9hw4dwjHxJAqRFe/iYXfboVFqUMuxHkBKhRIZeumDSy12CzRxGvTI9GPLxOFg2YlTp9if09Ki403ihx9Y12ONhm1/tSUmE9tqyctjAYh4fNvpDG1xcUPiUfZAtnzUavbcUEyH53kWBMqRWRIDdX+aN1IDREJikuQAaPbs2dixY0eT+7///nvMlnIEl/jN5rB5miCK2Z9AewBVWCpQnFqM3MTc5h8kCKyp4YkTrN4jKSm63iDE7M+kSUBuC19HtDGb2ZtyXp739zMhgWUgLJbwrMPpDG4LMzGRXUNOYvAnV22ROB6jte+peFqM6n8IiTmSA6CffvoJI0aMaHL/sGHDsHfvXjnWRBrhXTzsLjvUKjWq6gKv/+GdPCAAvbJ6NT/2wulk3ZxPnmQ1GWlpwQ2YlNuhQ8Bnn7EswV/+EunV+K+ujq05L69p7U3DLJDLFdp12O31s70C1XA4qlzk2v5qSNwabel7Kh67p/ofQmKO5ABIoVDAbDY3ud9oNMIV6n+8YxTn5CAIApQKpacLdCD1P5WWSnRM6YhOKZ18P8BiYdtd1dUsQ5GYGH19UcSTX2PHAl26RHYt/rJYWKYhL6/5zEtCAsu0hToLFOwxc6A+gJJzG4zn5a/DiY9v/XtKDRAJiVmSA6ALL7wQy5Yt8wp2XC4Xli1bhpEjR8q6OMLI0QRRHHvRJ7uP762zmprINjX0x5kzwIYN7HZbGXpqtbLtnby8louclUr2fXe7Q5sFcjiCL7YWu0Lb7fKsKZCmjP7wZ0gqNUAkJGZJ3tt4/PHHceGFF6J79+644IILAAD//e9/YTKZ8OWXX8q+QCJPAFRlqUJOYg6KU4ubflKc5aVWh66/ixxefpm9gQ8bBgwaFOnVtM5mY9/bvDxWj9KaxMT6jIU/j5fK6Qy8y3JjOh3L1sgxHFXclgtFL6eEhJaHpFIDREJiluQMUK9evbBv3z78+c9/RlVVFcxmM2bOnIkDBw6gT58+oVhjzDPzZsQpWawaSA8gz9iL7GbGXtjtLLCIVEdnfxgMwJtvstttIftjs7Hva26u/8f0FYrQZoHErtNy/D1rtexDjm2wYE6ltUbMrNntTcdjUANEQmJaQNWt+fn5ePTRR+VeC2mG2AQx0B5A4tiLkvRmhtU6HOFvxCfVG2+wzEjPnsAll0R6NS3jOPamnpfHtmCkaFgLJHcWSOyyLMd2j1LJ1ifHcNRgT6W1JjHR95BUaoBISEyT/I73yiuvYP369U3uX79+PV577TXJC1i5ciWKioqg0+lQWlqKXbt2tfj4FStWoHv37tDr9SgoKMCdd94JjuO8HiP1mtFMEAQ2BkOpDrgHUI2tBr2yejU/9oLnozv44Thg9Wp2e9686K7X4Hm23txclnmQKlRZILm6LDek17P1Sh082pDTyU4ZhnIbquGQ1IYcjvqtPEJIzJH8rrds2TJkZmY2uT87O1tyVmjdunUoKyvD4sWLsWfPHvTv3x9jxoxBVVWVz8e//fbbuP/++7F48WL8+uuvWL16NdatW4cHHngg4GtGO7vLDrvLHnAPICNnZGMv0psZeyEIrD4imn8LXr+enUzr0AG4+upIr6Z5PM++lzk5gQU/IjELVFcn39rE4aByBhridlrjwEIK8fh7qH/+xCGpNlv9fdQAkZCYJjkAOnHiBIqLmxbSFhYW4sSJE5Ku9dRTT+Gmm27CnDlz0KtXL6xatQrx8fFYs2aNz8fv2LEDI0aMwPTp01FUVITLL78c06ZN88rwSL1mtAu2B1ClpRLdMrohI76ZjJHTybZGoqnXT0MuF7BqFbt9yy3RW69ht7Ntq5wclm0IJkslZoEEQb4skN0uX5dlkTgcNZg6ILudbaWFOqsnDkkVAyBqgEhIzJMcAGVnZ2Pfvn1N7v/555+RkeH/tozdbsfu3bsxevTo+sUolRg9ejR27tzp8znnn38+du/e7Ql4fv/9d2zevBlXXHFFwNcEAJ7nYTKZvD6ihc1h82SApPYAstgt0MZp0T2ze/MPcjjqm8FFo82bgWPH2JvXtGmRXo1vDkd98JORIc+beUICCwzkyAK5XGxNoTjhFx9ff4xdKrebrStcp7CSklimiefrT8RFc+aTEBJSkgOgadOm4bbbbsNXX30Fl8sFl8uFL7/8ErfffjumTp3q93XOnj0Ll8uFnBzvbEZOTg4qKip8Pmf69Ol4+OGHMXLkSKjVapSUlGDUqFGeLbBArgmwbb2UlBTPR0FBgd9fR6jxLh5OtxNqpVpyAbQ49iIvMa/5B4m9XKKxrkYQ6sdezJkT3HaFIIRm1pbTyUZcZGUBmZnyfR8bZoGCHTsRii7LInH7KpCeQOJQ1nAFQOKQVKu1vgA6WgN/QkjISQ6A/v73v6O0tBSXXnop9Ho99Ho9Lr/8clxyySX4xz/+EYo1emzbtg2PPvoonn/+eezZswcbNmzApk2b8Pe//z2o6y5YsABGo9HzcfLkSZlWHDzOyUGAAIVCISkA8mvsBcDqN6K1APrbb4F9+9gb5A03BHcto5EFKufOAbW1bCip1creuAMNjJxOdh25gx9RfDx7ww62OzTPs2xSKP6exeGogQZAoVpXc8QhqeKJsGgM/AkhYSG58EOj0WDdunV45JFHsHfvXuj1evTt2xeFhYWSrpOZmQmVSoXKykqv+ysrK5HbzIDLhQsX4vrrr8eNf0wA79u3LywWC26++WY8+OCDAV0TALRaLbRR2gPH5rABf+wuSOkBJI69KEhpIZslFkBH62/BYvZn2jRWVxMosY4mP5+92TqdLPCz2VgmwGqt38KJi2MfYn+Y5t4gXS4W/GRmsgAoFG/iYidjo7H+tJRUcg8Z9SUxkQWVUojbZuFuvKnXs62wmhpqgEhIjAv4X+2uXbti8uTJuOqqq5CWloYXXngBQ4YM8fv5Go0GgwcPxtatWz33ud1ubN26FcOHD/f5HKvVCmWjNxrVH0dYBUEI6JrRjnNyUCgUknoAOd1O2Jw29Mnu42mg6JPDUV8LEW1++gnYvp0FIrfcEty1xN/2k5PZm19aGuvRU1wMFBWx/xYU1I+rUChYRsNorM8Ymc31AZPTyT6XkRG64EckZoECrQUK5faXSByOKmWrLtzbXw2lprK/52j8uSeEhE1QR3+++uorrFmzBhs2bEBKSgomTpwo6fllZWWYNWsWhgwZgqFDh2LFihWwWCyYM2cOAGDmzJno0KEDli1bBgAYP348nnrqKQwcOBClpaU4fPgwFi5ciPHjx3sCodau2daY7awLtJQeQNWWauQm5qIotajli4sF0I2nk0ea3Q7ccw+7fe21LDgJ9nrZ2U2zOQoFexNsPAJErLsRgx27vb6/j1hAm57OrhnqHjJiFshkCiwLxPMsSxXKdWo09V2h/V0fz7OvKxKnD+Pj2feEMkCExDTJ//qcPn0ar776Kl555RUYDAbU1tbi7bffxp///OeWa018mDJlCqqrq7Fo0SJUVFRgwIAB2LJli6eI+cSJE14Zn4ceeggKhQIPPfQQTp8+jaysLIwfP96r9qi1a7Y1Jt4EjdL/HkBuwQ0jb0Rpx1Lo4lr5B97hYG/20VYHsXIl8OuvLMhYuDC4a/E8e4OWUkDdMDBqSBDqgyKtNnwN9MTsldEorbN0qIaMNqZQsPWVl/sfTLtckQu8xWGuhJCYphAE/86vvv/++1i9ejW+/vprjBs3Dtdddx3GjRuHhIQE/Pzzz+jVq1eo1xo2JpMJKSkpMBqNSA7FUEo/uQU33tz3JpwuJ/6v+v/w5HdPom92X/zjkuaLzc9az0KAgMm9JiNJ28o/8uXl7E01gl9jEwcOAGPHskDj+eeBa64J7noGQ/2WV1tmtQLHj7Ogwd+sCcexGqCiotBnWmw24OhR/4qa7Xb2UVREx9AJIbKS8v7td/HClClTMHDgQJSXl2P9+vW45pproKF/vEKKd9Y3QfS3B1CNrQY9M3u2HvxEYwG00wncdRcLfi6/PPiuz243+zrbw2/7er30E2Ecx772cGwzSRmOyvPs66F/PwghEeR3ADR37lysXLkSY8eOxapVq1Ar9dQHkYxzcnC4HF5jMFoKgIycEYmaRHTNaGbsRUPiVk40dYB++WVg716WRVi2LPitOXH6eXsYdyDWAgHs76414dr+EonDUf0JgByO9hGUEkLaNL8DoBdffBHl5eW4+eabsXbtWuTl5eGaa66BIAhwh6LBHAHn5FgGyM8miFXWKnRN74rM+Kaz2pqw26PrBNjvvwNPPMFuL1rEhokGi+NY0BCtfY6kkpIFsttZhiWchb56fetdoUMxlJUQQgIg6Z1Br9dj1qxZ2L59O3755Rf07t0bOTk5nvlcGzZsCNU6YxLn5FgXaJW61R5AVocVaqUaPTJ7+Hdxf7II4eJ2s1NfHAdccAEgoaN4sxwOlt2KthNuwRCzQEpl639/HMe+9nAGuOJw1JayQOKx/Cjtu0UIiR1B9QF69NFHcfLkSbz55puwWq2YFq2zmtoo3sUDCvjVA6iijo29yE/K9+/iVmv4TjG15o03gO++Y9s1Tzwhz6k0m40FAO3tjdbf7tAuF2tQGE5iwNlaACT3UFZCCAlA0HsDSqUS48ePx8aNG6NqhER7IHaBbq0HkN1lh1two1d2K2MvRG43yxBEw/bX6dOA2MZgwYLge/4A9X18UlLa5xtta1mgSGx/iRISmp9gH46u1IQQ4idZiyOys/0b0kn8Y3PavGaANdcDqKKuAh2TO6JTSif/LhwtE+AFAbj3XpbNOO88YPZsea7bnoqffWmtFojn2dceiVNWLQ1HDUdXakII8VM7qQ5tn4ycEVqVFlV1zdf/uNwu/8ZeNGS3s9/SI30CbP16YNs2tk31z3/KV6zMcexEUqS/vlASs0C+Ao1InrLSaFiA5msbjOfZtly0bL0SQmIaBUBRrM5ehzhlXIs9gKosVchNzEVxWrH/F46GAuiqKmDpUnb7rruALl3kua7Lxba9wl3/Em56PQuCGmeBxMxeJLMsSUlNf8bCfSyfEEJaQQFQlHK5XbA4LC32AHILbhh4A3pn9W597EVDVmvksyMPPsi6NPfrF/yw04bEwaexUGeSksL+HhtmgaKhyaBOx9bVcDhqJOuSCCHEB8kBUOfOnVFTU9PkfoPBgM6dO8uyKOJfE8RaWy0y9Bnoki4he+JysTfJSNb/fPIJsHkze5P85z/lDcbsdpYZaY/Fz435ygLZ7Wz7L5JfvzjlveE2mFiXFOm6M0II+YPkAOjYsWNw+TjlwfM8Tp8+LcuiyB9NEN1/NEFspgfQWdtZ9MzyY+xFQ5EugD53jmV/AGD+fKB3b/muHcjg07auYRZI7Owd6SyLOGy0YWbK6Wz/25KEkDbF71+9P/roI8/tTz/9FCkpKZ4/u1wubN26FUVFRbIuLpaJGaA4ZZzPDJAgCFBAgdxEiR2TxREYkSpEXbwYOHsW6NYNuO02ea9ts7HBp7E0Y0qnY1mgs2frt5iiofeRXs8CIbebZR0jXZdECCGN+B0ATZgwAQCgUCgwa9Ysr8+p1WoUFRXhySeflHVxsYxzcnC5XTDZTT57ANlddmjjtEjSSDztw/ORGw2xdSuwYQN7/SeflPeNuj0NPpUqJYXVU9lsQEZGdGz/NewK7XLVH48nhJAo4XcAJM77Ki4uxg8//IDMTD/mTZGAcU4OUKDZHkA2pw26OB0SNRK3FWy2yBRAm83Affex2zfdBAwaJO/123vvn5aIWaDa2ugp/lYqWTBaXc3+nJUVHYEZIYT8QfI74dGjR5vcZzAYkCpOqiay4F08IKDZHkA2hw3JumTo1RLe8MQC6EgEQI88ApSXA0VFbO6X3DgOyMtrP4NPpRL//xcN218iMRhVKGj7ixASdSS/Wzz++ONYt26d58+TJ09Geno6OnTogJ9//lnWxcUyM2+GSqFqtgcQ5+T8m/rekN3OaoDCvRXx7bfAm2+y2//8p/xZivY4+FQqrRbIyYmuAFDcBhM/CCEkikj+13LVqlUo+GNe0+eff44vvvgCW7Zswbhx43BPKH6zj1F19roWj8A73A6k69OlXdThYFmgcL5JWq31GZ/rrweGD5f/NWw2dsKIsgzRRQxKk5OjKzAjhBAEsAVWUVHhCYA++eQT/PnPf8bll1+OoqIilJaWyr7AWGXiTVCr1C1OgZdcAG23h78OY/ly4PhxID+//vi7nMTBp8nJ8l+bBC8zk2p/CCFRSfKvZWlpaZ6p71u2bMHo0aMBsGPZvvoDEekcLgc4J8cCIB89gBwuB9RKtfQCaIslvNtfu3cDL7/Mbj/+eGhOaMVy8XNboFZHvus4IYT4IPlfpmuvvRbTp09H165dUVNTg3HjxgEAfvrpJ3SRa55TjBObICbEJfjMANmcNujUOmkNEJ1OlgEK15sRz7MZX4IA/OlPwCWXhOZ1OA7IzqY3WUIIIZJIftd4+umnUVRUhJMnT2L58uVI/KO7a3l5Of7617/KvsBYJDZBtApWnz2AOCcHfZwe8WoJWQ+xA3S4jkk/8wxw6BA7/rxkSWheQxx8GsvFz4QQQgIiOQBSq9W4++67m9x/5513yrIg8kcGyGWHxc5mPDXpAeSwIS8pD0qFhB1Mu51lY8JRjLp/P/Dcc+z2P/7BujOHQiwNPiWEECKrgN4N33jjDYwcORL5+fk4fvw4AGDFihX48MMPZV1crOKcHAQIqLawJnKNewDxLh6ZeolH4Hk+PMWoDgfb+nK5gCuuAK68MnSvFUuDTwkhhMhKcgD0wgsvoKysDOPGjYPBYPAUPqempmLFihVyry8mcU4OAJrtAQQAyTqJp56s1vAMQF21imWAUlNZ9idUYnHwKSGEENlIDoD+9a9/4aWXXsKDDz4IVYOBmkOGDMEvv/wi6+JiFefkoBAUPgugXW4XlAqltBNg4ZoAf/o08PTT7PbSpaw4OVRsNnaqjOZLEUIICYDkAOjo0aMYOHBgk/u1Wi0sFossi4p1Jt7U7BR4m9MGvVovPQCy20MfAD33HMvMDB8OTJoUuteJ5cGnhBBCZCE5ACouLsbevXub3L9lyxb07NlTjjXFPLPd3GwPIM7JSR+C6nCwgCGUtTKnTwNr17Lbd90V2tfiOFb4TNtfhBBCAuT3KbCHH34Yd999N8rKynDrrbeC4zgIgoBdu3Zh7dq1WLZsGV4Wm96RgAmCADNvbjYDxDk55CXlIU4p4QAfxwENtitDYuVKFmgNHx6acRcNxfrgU0IIIUHz+1106dKl+Mtf/oIbb7wRer0eDz30EKxWK6ZPn478/Hw888wzmDp1aijXGhPsLjscbgesdt89gGxOm8+i6GYJQugLoM+cqc/+lJWF7nWA+lom6v1DCCEkCH4HQIIgeG7PmDEDM2bMgNVqRV1dHbJDWewaY8QeQAbeAKBpDyABApK1Ek6AOZ2hnwC/ciWrMRo2DDj//NC9DkCDTwkhhMhCUiNERaO6jvj4eMRTHYasOCcHu9OOWq4WgHf9j1twQwGFtCGodjsLgEKVMSkvB95+m90OdTNMQWD9hWjwKSGEkCBJCoC6devWJAhq7Ny5c0EtKNZxTg5OwYkaaw0AICs+y/M53slDG6eNrgLo559nQVZpKTBiRGheQ8RxgFZLxc+EEEKCJikAWrp0KVJSUkK1FoL6LtDNHoGPk3gEPpQF0BUVwFtvsdt33hn6jsw0+JQQQohMJL2TTJ06lep9QkzsAi0GQI2PwKfr06GN0/p3MbEAOlT1P88/z/r+nHceMHJkaF5DRINPCSGEyMjvc8StbX0FY+XKlSgqKoJOp0NpaSl27drV7GNHjRoFhULR5OPKBjOnZs+e3eTzY8eODdn65WRz2FgXaKvvDJCkE2BiB+hQZEwqK+uzP2Vloc/+2Gws+KHBp4QQQmQQ0CkwOa1btw5lZWVYtWoVSktLsWLFCowZMwYHDx70mW3asGED7Ha75881NTXo378/Jk+e7PW4sWPH4pVXXvH8Wav1M2sSYSbeBJVS1ewYjFRdqv8XczjYKbBQBEDPP8+2pIYMAS64QP7rN2a3s+0vGnxKCCFEBn5ngNxud0i2v5566incdNNNmDNnDnr16oVVq1YhPj4ea9as8fn49PR05Obmej4+//xzxMfHNwmAtFqt1+PS0tJkX3somOwmVgj9Rw+gzHg29V0QBAgQpNX/iIGi3EFDVRXw5pvsdjiyPzxPxc+EEEJkFdFWuna7Hbt378bo0aM99ymVSowePRo7d+706xqrV6/G1KlTkdCoNmTbtm3Izs5G9+7dMW/ePNTU1DR7DZ7nYTKZvD4iwS24YbFbYOSMAIBMfX0PILvLDq1K4gkwmy00BdBi9mfQIODCC+W/fmNi7x8afEoIIUQmEQ2Azp49C5fLhZycHK/7c3JyUFFR0erzd+3ahf379+PGG2/0un/s2LF4/fXXsXXrVjz++OPYvn07xo0bB5fL5fM6y5YtQ0pKiuejoKAg8C8qCLyTh8PlgJFnAVDjERj6OD2StH72AHK7WeAgdwfo6mrgjTfY7XBkf9xu9l8afEoIIURGbfo88erVq9G3b18MHTrU6/6GIzn69u2Lfv36oaSkBNu2bcOll17a5DoLFixAWYMRDiaTKSJBEOfk4HA7cM7Geik1LoBOUCdAH+dnEbBY/yN30fALL7Dsz8CBwKhR8l7bF45jXZ9p+4sQQoiMIpoByszMhEqlQmVlpdf9lZWVyM3NbfG5FosF77zzDubOndvq63Tu3BmZmZk4fPiwz89rtVokJyd7fUSCOAajxsa26xoHQFkJWf6fxgvFCbCzZ4HXXmO3w5H9AVgAlJJCg08JIYTIKqLvKhqNBoMHD8bWrVs997ndbmzduhXDW5kovn79evA8j+uuu67V1zl16hRqamqQl5cX9JpDSSx+Pms9C8C7B5DD5UBGfEZzT22qwUk52YjZnwEDgIsvlv/6jdHgU0IIISES8V+ry8rK8NJLL+G1117Dr7/+innz5sFisWDOnDkAgJkzZ2LBggVNnrd69WpMmDABGRneQUFdXR3uuecefPfddzh27Bi2bt2Ka665Bl26dMGYMWPC8jUFinNygAI+j8ADkF4ALWf9T8PsTzi6PgOAxUKDTwkhhIRExGuApkyZgurqaixatAgVFRUYMGAAtmzZ4imMPnHiBJSNtj8OHjyIb775Bp999lmT66lUKuzbtw+vvfYaDAYD8vPzcfnll+Pvf/971PcC4pwc3G53kwDI4XJApVT5HwC53SxTI+f214svsqCqf3/ARx2V7JxOFmSlpob+tQghhMSciAdAADB//nzMnz/f5+e2bdvW5L7u3bs325hRr9fj008/lXN5YWNxWGB1WOFwO7x6AHFODnq1hBlgck+Ar6kBXn2V3Q5X9qeujgU/VPxMCCEkBCK+BUbqmXgTTDzrQdSwBxDn5BAfF+9/AORwsNlZcvUAevFFNlOsXz+gQc+mkHE4WNFzaip1fiaEEBISFABFETNv9jRBbHwCLF2fDqXCz78uOQugz50DxJEi4cz+pKRQ9ocQQkjIUAAUJZxuJ2wOGwy8AYB3AMQ7ec92mF+sVvkKoMXsT58+wGWXyXPNltjtrHaJan8IIYSEEAVAUYJ38rC77T6bIAoKwf8O0C4Xm50lRwF0w+xPuPr+iLU/NPWdEEJICFEAFCU4JweHq74LtNgDyOV2QQEFkjR+BkBiA0Q5MkD//jc7it67N3D55cFfrzU8z+Z9paSE/rUIIYTENAqAooQ4BqPaWg2gPgPEu3jo4ySeAHO7gy+Arq0Nf+2PxcKyP9T3hxBCSIhRABQlxDEY1RbvAMjmsEkPgOTw0ktsO6pXLyAcDSQ5DtBqKftDCCEkLCgAihKck4PFYWnSA8jmtCFVlwq1ys8tLTkKoGtrgTVr2O077wzPHC6rFUhLY0EQIYQQEmIUAEUJzsnhnJXV/zTsAWRz2pCZ4OcJMKeTZYCCDYBefhkwm4GePYGxY4O7lj9sNrbtFaEhtIQQQmIPBUBRos5eByPftAeQIAhI0fq5LeRwBB8AGQzA6tXs9h13hCf7Y7Ox7I9GE/rXIoQQQkABUNQw8aYmTRAFQYACCmkdoN3u4IKW1atZ9qdHD+CKKwK/jr9sNnbknbI/hBBCwogCoChh5s2o5WoBeJ8A08Zp/e8BxPPBBT9GI9v+AsKT/REEFgClp8s7uZ4QQghpBQVAUcDhcoBzcai1sQBI7AFkc9igi9P5nwGyWIILJFavBkwmoHt34MorA7+Ov8TsT5KfAR4hhBAiEwqAooDYBLHGVgOgwRF4pw1J2iTo4vzoi+N0si2wQDtAm0z12Z/bbw9P9ofjgIwMebpWE0IIIRJQABQFOCcHzsnhrPUsgPoAiHNy/s8As9uD6wC9ejXbAuvWDbjqqsCuIYXVCiQkUPaHEEJIRFAAFAU4J9v+atwDyOl2Il2f7t9FgimANplY40OA1f4E20W6NW43q1dKTw/9axFCCCE+UAAUBTgnhxqObX817AEkQPC//ieYAug1a1j2p0uX8GV/EhPZByGEEBIBFABFAc7JodbqfQLM7rJDo9T4FwAJAgsqAumj43DUd30OV/bHbqfsDyGEkIiiACgKcE4O5zjWBbph/Y9erfdvCrzYATqQYuKvvwZqaoDMTGD8eOnPl8piYXU/lP0hhBASQRQARQFfTRCtDivi1fGIV8e3fgGHI/AC6A8+YP+9+urQn8ZyuViwlp4eng7ThBBCSDPoXSgKNGyCKPYAEk+AKRSK1i8gToD357ENWSzAli3s9sSJ0p4bCIuFdXxOSAj9axFCCCEtoAAowgRBgNluxjmb9xaY3WVHRnyGfxfhuMAyKp9+ypoRFhUBAwdKf74ULher/0lLo+wPIYSQiKN3ogjjXTyrAWoUAEEB/+p/xALoYLa/Jk6Unj2Sqq6Osj+EEEKiBgVAEcY7edRYa7x6ADndTqgUKv9OgDkcrK5GagB09iywfTu7HertL6eTBWppaaEPtAghhBA/UAAUYTanDZWWSgD1PYA4Jwd9nN7/ACiQERgff8y2pfr3B0pKAli5BHV1QEoKEO9HQTchhBASBhQARRjv5FFtqQbQYAaYlCGoDgfLrkjNrGzYwP4b6uyPw8HWRtkfQgghUYQCoAhrrgdQuj7d0xG6RTab9OzPsWPAnj2sGPmaaySuWCKLBUhNZVPfCSGEkChBAVCEiXPAgAYBkMvPIaiBFkCLxc8jRwLZ2dKeK4XdzoKs1FTK/hBCCIkqFABFmNVh9WSAxB5AgiAgWZvc+pPFAmgpGSBB8D79FUqU/SGEEBKlKACKMDNvhsFmAMAyQG7BDYVCgSStH0fg7XbpJ8B++QU4cgTQ6YBx4wJbtD94ngVmqamhew1CCCEkQBQARZiBN3jVAHFOTnoBtBRi8fNll7GZXKFitbLgR6cL3WsQQgghAaIAKILcghvl5nI43U5PDyCb0+bfEXhBYMfLpWR/XC7go4/Y7WuvDXzhreE4ti7K/hBCCIlSFABFEOfkUFFXAaBBDyAHh2RtMjQqTctPtloBs1lab51vvwUqK1lgMmpUwOtuldXKjr1rtaF7DUIIISQIFABFEOfkPE0QGx6Bz0rIavmJggAYDOxklcqPo/Iisfj5qqsATSsBVqB4ngU+KSmhuT4hhBAig6gIgFauXImioiLodDqUlpZi165dzT521KhRUCgUTT6uvPJKz2MEQcCiRYuQl5cHvV6P0aNH49ChQ+H4UiThnByqLFUA6gMgl+BCqi615SdarYDRKG2uls0GbN7Mbody+4vnWVYqVAEWIYQQIoOIB0Dr1q1DWVkZFi9ejD179qB///4YM2YMqqqqfD5+w4YNKC8v93zs378fKpUKkydP9jxm+fLlePbZZ7Fq1Sp8//33SEhIwJgxY8BxXLi+LL9wTg5nrWcBsABI+KOgucX6HzH7A0g7/v7FF6xmqEMH4LzzAlyxH5xOINGPAm5CCCEkgiIeAD311FO46aabMGfOHPTq1QurVq1CfHw81qxZ4/Px6enpyM3N9Xx8/vnniI+P9wRAgiBgxYoVeOihh3DNNdegX79+eP3113HmzBls3LgxjF9Z63gnj1qONUHMSciB3WWHNk7b8hR4mw0wmaQHGQ17/yhD9NfucLDiZzr5RQghJMpFNACy2+3YvXs3Ro8e7blPqVRi9OjR2Llzp1/XWL16NaZOnYqEP7aDjh49ioqKCq9rpqSkoLS0tNlr8jwPk8nk9REONqfNqwu0zdnKDDAx+yMI0rI/tbXAl1+y26FsfsjzLPih7S9CCCFRLqIB0NmzZ+FyuZCTk+N1f05ODioqKlp9/q5du7B//37ceOONnvvE50m55rJly5CSkuL5KCgokPqlBKTOXufJAGUnZMPmsCFBkwBdXDMZFJtNeu0PAGzaxLIzPXsCPXoEueoWOBystxCNvSCEEBLlIr4FFozVq1ejb9++GDp0aFDXWbBgAYxGo+fj5MmTMq2wZSeMJ7x6AHFODlnxWVA0F0AYjSz7E+jsr1AWP7vdLPCh7S9CCCFtQEQDoMzMTKhUKlRWVnrdX1lZidzc3Bafa7FY8M4772Du3Lle94vPk3JNrVaL5ORkr49wOFp7FEB9DyCH24F0fbrvBwea/Tl9GvjuOxachHLyu3j8nQIgQgghbUBEAyCNRoPBgwdj69atnvvcbje2bt2K4cOHt/jc9evXg+d5XHfddV73FxcXIzc31+uaJpMJ33//favXDCen24nTptMA4NX3p9kCaIOBZVmkZn/Ewu9hw9gJsFDhOLb9FaoCa0IIIURGEippQ6OsrAyzZs3CkCFDMHToUKxYsQIWiwVz5swBAMycORMdOnTAsmXLvJ63evVqTJgwARkZGV73KxQK3HHHHXjkkUfQtWtXFBcXY+HChcjPz8eECRPC9WW1inNyqLTWN0F0uByIU8b5LoAWsz9Suj6LwjH5XZxHFsj6CCGEkAiIeAA0ZcoUVFdXY9GiRaioqMCAAQOwZcsWTxHziRMnoGyUVTh48CC++eYbfPbZZz6vee+998JiseDmm2+GwWDAyJEjsWXLFuiiaHuGc3KotlQDYEfgbU4b9Gq97ynwRiOb4yX1dNWvv7IPjQZo0ChSdnY7e40o+v4SQgghLYl4AAQA8+fPx/z5831+btu2bU3u6969u6dpoC8KhQIPP/wwHn74YbmWKLvGTRA5Jwd9nB7x6kZZFI4LrPYHqM/+XHJJaAeT8jzb/pJyNJ8QQgiJICrYiBDOyeGc7RyA+gAoIz4DSkWjvxKjkXVXlpr9cbvDs/0FUPdnQgghbQ4FQBFic9i8egBxTg6Z+kzvB3EcK34OpLZm1y7gzBmWmWnQFFJ21P2ZEEJIG0QBUIScNp/26gEEAMm6RsfvTSYWYGi10l9gwwb23yuuCG1wQt2fCSGEtEEUAEXI7+d+BwBk6DOggAJKhdL7BBjPs+xPILU/djvr/gyEfvvL4QCSk6n7MyGEkDaFAqAI+d3AAqBmZ4AFk/356isWPOXkAOefL8+CfRG7PweyRkIIISSCKACKkJNGNm7DcwJMra8PgHieDTDV6wO7uLj9dc01gEolw2qbQd2fCSGEtFEUAEWA3WVHpYU1QcxJyAHn5JCqS0Wc8o9j5GYz28YKJLAwm4EvvmC3Qzn7C6Duz4QQQtoseueKgIZNEMUtsOyEbPZJu51lfwLtqrx5MwtMunQB+vSRacU+UPdnQgghbRgFQBHAOTmctdU3QRQEAcnaP06AmUz1J6sC0bD3TygLk6n7MyGEkDaMAqAIsDlsniaIWfFZUCgUbAiqmP0JtPanshL49lt2O9Snv3ieZX+o+zMhhJA2iAKgCDhlOuXpAZSoSYQ2TssKoM1mFlgEGgB9+CE7mTV4MFBYKO+iG6Puz4QQQtowCoAi4KjhKADWA8jpdkIfp0eiQhtc9geo3/4KdfEzdX8mhBDSxlEAFAHHao8BqC+ATtIkQWvlWfFyoAHQ4cPAvn3s2Pv48fIt1hfq/kwIIaSNowAoAo4ZjwGoD4ByNOnyZX8uugjIyAh+kS2h7s+EEELaOAqAIkBsgpiTkAOX24V0p5plfwLdUhKE8G1/UfdnQggh7QAFQGEmCALK68oBsBNgcDqRZHGy7E+gGZU9e4Djx9mprDFjZFytD9T9mRBCSDtAAVCY8S7e0wQxXZ+OJB6IdyuDCyjE7M/YsaFvTEjdnwkhhLQD9C4WZjaHDTW2GgBASlwikq0uJCZlBJ79cTiAjz5it0Pd+4e6PxNCCGknKAAKs5Omk3C4HVBCiWSHEolOFXRJaYFf8L//BWpqWOHzhRfKt1BfqPszIYSQdoICoDA7cu4IACAjPh3K2lqkpuZAEcx2krj9dfXVoe/KTN2fCSGEtBMUAIXZ74bfAQA5ukzAZkNKWm7gF7NagS1b2O1Qb38B1P2ZEEJIu0EBUJidMJwAAOSp0uCKi0O8JiHwi336KQuCioqAQYPkWWBzqPszIYSQdoQCoDA7bT4NAMhRJUOI1yEhmABowwb23wkTQt+UkLo/E0IIaUcoAAqzcvMZAECaNhU6tR4J6gADoJoaYPt2djsc21/U/ZkQQkg7QgFQmJ0xsQxQsj4V+jh9YBkgQQDefBNwuYB+/YAuXWReZSPU/ZkQQkg7Q8d5wkgQBJRbKgAAyeoEpOpSoVRIjEF/+QX4+9+Bb79lf/7Tn2RepQ/U/ZkQQkg7QwFQGFVaKsG77FBAgUSlHun6dP+ffPo08Nhj9XU/Wi0wdy4wa1ZoFtsQxwFZWdT9mRBCSLtBAVAYHTMcAwCkxyVCqVQiUevHkXKTCXjuOeDll1kmBmADT++7D+jYMXSLFVH3Z0IIIe0QBUBhJAZAGXEpUEDRcgG03Q688Qbw9NNAbS27b/hwYOFCoH//0C+24Tqo+zMhhJB2hgKgMBIDoDRVAnRxOt8BkCAAmzcDjz4KHGOPR5cuwIMPApddFv5TWDzPhp9S92dCCCHtCL2rhVHf7L6YWHwFEkw2FgA1PgH244+swPnHH9mfs7KAu+4Cpk2LXABC3Z8JIYS0QxQAhdGV3a5EJyEZX+9ci2RtMtQqNfvE0aPAsmXApk3szzod8Je/APPmRTb4oO7PhBBC2ikKgCIkIz4DOHcOWLECeP11FmwoFMCUKcDddwN5eZFeInV/JoQQ0m5RABQBOpcShW9tBl5Zx055AcDFF7M6n549I7u4hhwOtg1H3Z8JIYS0MxFv7LJy5UoUFRVBp9OhtLQUu3btavHxBoMBt956K/Ly8qDVatGtWzds3rzZ8/klS5ZAoVB4ffTo0SPUX4Z/3G6kfvwZptz3BnKeeYkFP716AWvXss7O0RT8UPdnQggh7VhEM0Dr1q1DWVkZVq1ahdLSUqxYsQJjxozBwYMHkZ2d3eTxdrsdl112GbKzs/Hee++hQ4cOOH78OFJTU70e17t3b3zxxReeP8dFywmmW25BwcsvAwDcOdlQ3r8AmDQJUKkivDAfqPszIYSQdiyikcFTTz2Fm266CXPmzAEArFq1Cps2bcKaNWtw//33N3n8mjVrcO7cOezYsQNqNSsgLioqavK4uLg45ObmhnTtAZk1C661b+PA1eejZPEK6JLSIr2i5vE8kJlJ3Z8JIYS0SxF7d7Pb7di9ezdGjx5dvxilEqNHj8bOnTt9Puejjz7C8OHDceuttyInJwd9+vTBo48+CpfL5fW4Q4cOIT8/H507d8aMGTNw4sSJkH4tfhs5Ege2v49TU66I7uBHENgHdX8mhBDSTkUsA3T27Fm4XC7k5OR43Z+Tk4MDBw74fM7vv/+OL7/8EjNmzMDmzZtx+PBh/PWvf4XD4cDixYsBAKWlpXj11VfRvXt3lJeXY+nSpbjggguwf/9+JCUl+bwuz/PgxTETAExiYXIIKJKTkZmQGbLry4K6PxNCCGnnoqQ4xj9utxvZ2dn497//DZVKhcGDB+P06dN44oknPAHQuHHjPI/v168fSktLUVhYiHfffRdz5871ed1ly5Zh6dKlYfkaumV0g5uL8swKdX8mhBDSzkVsCywzMxMqlQqVlZVe91dWVjZbv5OXl4du3bpB1aBouGfPnqioqIDdbvf5nNTUVHTr1g2HDx9udi0LFiyA0Wj0fJw8eTKAr8g/cco4aJTqkF1fFtT9mRBCSDsXsQBIo9Fg8ODB2Lp1q+c+t9uNrVu3Yvjw4T6fM2LECBw+fBhut9tz32+//Ya8vDxommnWV1dXhyNHjiCvhcaCWq0WycnJXh8xi7o/E0IIiQERPeJTVlaGl156Ca+99hp+/fVXzJs3DxaLxXMqbObMmViwYIHn8fPmzcO5c+dw++2347fffsOmTZvw6KOP4tZbb/U85u6778b27dtx7Ngx7NixAxMnToRKpcK0adPC/vW1SdT9mRBCSAyIaJHHlClTUF1djUWLFqGiogIDBgzAli1bPIXRJ06cgLLBMeyCggJ8+umnuPPOO9GvXz906NABt99+O+677z7PY06dOoVp06ahpqYGWVlZGDlyJL777jtkZWWF/etrk6j7MyGEkBigEARBiPQioo3JZEJKSgqMRqP822EGA3D6NJAWhcfg3W7WnbqoiI7AE0IIaXOkvH9TlztSj7o/E0IIiREUAJF64vF36v5MCCGknaN3ukgQOy1HE+r+TAghJIZQABRu8fGsx05tLau5iRbU/ZkQQkgMoQAo3DQaID8fSE5mBdHREgTxPAvOqPszIYSQGEABUCRoNEBeHpCayoKgRsNcw04QqPszIYSQmEK/7keKWg3k5rKC43PnWEYoEtkXp5MdfU9MpPofQgghMYMCoEiKiwNycljTwZoadgJLHcY5YTYb+8jIYM0PafuLEEJIjKB3vEhTqVgQpFQC1dUsExPqMRSCAJjNLPDq0IFtxVHnZ0IIITGEAqBooFSyDIxSCVRVsQBFqw3Na7lcgNEIJCQA2dnsv4QQQkiMoQAoWiiVQGYmy8SIQZDcR9I5DrBagfR0FnCFc7uNEEIIiSIUAEUThYLV46hUQHk5C4L0+uCvKwhAXR37b14em0NG3Z4JIYTEMAqAoo1CwQIUhQKoqAi+O7O45aXXs1ojOupOCCGEUAAUtVJTWZamvJxlbwIJXHgesFhYQJWVFfriakIIIaSNoAAomiUn1wdBZjM7Ju+vujqW/cnNZTU/tOVFCCGEeNC7YrRLTGSjM1QqtpXVGrebzRlTqYCOHVlhNQU/hBBCiBd6Z2wLEhJYvx6Nho3OaG6SvN3OPp+SwoIfKRkjQgghJIZQANRW6PUsCNLrfQdBFgs74p6Tw056haqPECGEENIOUADUluh0bDssMZFtc7nd7MNgYKfGOnZkxc4qVaRXSgghhEQ1KoJua7RaluFRKlngA7Atr6ws+RsnEkIIIe0UBUBtkUZTP0k+Lo4VOlPWhxBCCPEbBUBtlVrNtsNoiCkhhBAiGdUAtWUU/BBCCCEBoQCIEEIIITGHAiBCCCGExBwKgAghhBAScygAIoQQQkjMoQCIEEIIITGHAiBCCCGExBwKgAghhBAScygAIoQQQkjMoQCIEEIIITGHAiBCCCGExBwKgAghhBAScygAIoQQQkjMoQCIEEIIITEnLtILiEaCIAAATCZThFdCCCGEEH+J79vi+3hLKADywWw2AwAKCgoivBJCCCGESGU2m5GSktLiYxSCP2FSjHG73Thz5gySkpKgUChgMplQUFCAkydPIjk5OdLLixn0fY8M+r5HBn3fI4O+7+EXyu+5IAgwm83Iz8+HUtlylQ9lgHxQKpXo2LFjk/uTk5Pp/yARQN/3yKDve2TQ9z0y6PsefqH6nreW+RFRETQhhBBCYg4FQIQQQgiJORQA+UGr1WLx4sXQarWRXkpMoe97ZND3PTLo+x4Z9H0Pv2j5nlMRNCGEEEJiDmWACCGEEBJzKAAihBBCSMyhAIgQQgghMYcCIEIIIYTEHAqAWrFy5UoUFRVBp9OhtLQUu3btivSS2rUlS5ZAoVB4ffTo0SPSy2p3vv76a4wfPx75+flQKBTYuHGj1+cFQcCiRYuQl5cHvV6P0aNH49ChQ5FZbDvS2vd99uzZTX7+x44dG5nFtiPLli3Deeedh6SkJGRnZ2PChAk4ePCg12M4jsOtt96KjIwMJCYmYtKkSaisrIzQitsHf77vo0aNavIz/5e//CUs66MAqAXr1q1DWVkZFi9ejD179qB///4YM2YMqqqqIr20dq13794oLy/3fHzzzTeRXlK7Y7FY0L9/f6xcudLn55cvX45nn30Wq1atwvfff4+EhASMGTMGHMeFeaXtS2vfdwAYO3as18//2rVrw7jC9mn79u249dZb8d133+Hzzz+Hw+HA5ZdfDovF4nnMnXfeiY8//hjr16/H9u3bcebMGVx77bURXHXb58/3HQBuuukmr5/55cuXh2eBAmnW0KFDhVtvvdXzZ5fLJeTn5wvLli2L4Krat8WLFwv9+/eP9DJiCgDhgw8+8PzZ7XYLubm5whNPPOG5z2AwCFqtVli7dm0EVtg+Nf6+C4IgzJo1S7jmmmsisp5YUlVVJQAQtm/fLggC+/lWq9XC+vXrPY/59ddfBQDCzp07I7XMdqfx910QBOGiiy4Sbr/99oishzJAzbDb7di9ezdGjx7tuU+pVGL06NHYuXNnBFfW/h06dAj5+fno3LkzZsyYgRMnTkR6STHl6NGjqKio8PrZT0lJQWlpKf3sh8G2bduQnZ2N7t27Y968eaipqYn0ktodo9EIAEhPTwcA7N69Gw6Hw+tnvkePHujUqRP9zMuo8fdd9NZbbyEzMxN9+vTBggULYLVaw7IeGobajLNnz8LlciEnJ8fr/pycHBw4cCBCq2r/SktL8eqrr6J79+4oLy/H0qVLccEFF2D//v1ISkqK9PJiQkVFBQD4/NkXP0dCY+zYsbj22mtRXFyMI0eO4IEHHsC4ceOwc+dOqFSqSC+vXXC73bjjjjswYsQI9OnTBwD7mddoNEhNTfV6LP3My8fX9x0Apk+fjsLCQuTn52Pfvn247777cPDgQWzYsCHka6IAiESVcePGeW7369cPpaWlKCwsxLvvvou5c+dGcGWEhN7UqVM9t/v27Yt+/fqhpKQE27Ztw6WXXhrBlbUft956K/bv30+1hWHW3Pf95ptv9tzu27cv8vLycOmll+LIkSMoKSkJ6ZpoC6wZmZmZUKlUTU4BVFZWIjc3N0Krij2pqano1q0bDh8+HOmlxAzx55t+9iOvc+fOyMzMpJ9/mcyfPx+ffPIJvvrqK3Ts2NFzf25uLux2OwwGg9fj6WdeHs19330pLS0FgLD8zFMA1AyNRoPBgwdj69atnvvcbje2bt2K4cOHR3BlsaWurg5HjhxBXl5epJcSM4qLi5Gbm+v1s28ymfD999/Tz36YnTp1CjU1NfTzHyRBEDB//nx88MEH+PLLL1FcXOz1+cGDB0OtVnv9zB88eBAnTpygn/kgtPZ992Xv3r0AEJafedoCa0FZWRlmzZqFIUOGYOjQoVixYgUsFgvmzJkT6aW1W3fffTfGjx+PwsJCnDlzBosXL4ZKpcK0adMivbR2pa6uzus3rKNHj2Lv3r1IT09Hp06dcMcdd+CRRx5B165dUVxcjIULFyI/Px8TJkyI3KLbgZa+7+np6Vi6dCkmTZqE3NxcHDlyBPfeey+6dOmCMWPGRHDVbd+tt96Kt99+Gx9++CGSkpI8dT0pKSnQ6/VISUnB3LlzUVZWhvT0dCQnJ+Nvf/sbhg8fjmHDhkV49W1Xa9/3I0eO4O2338YVV1yBjIwM7Nu3D3feeScuvPBC9OvXL/QLjMjZszbkX//6l9CpUydBo9EIQ4cOFb777rtIL6ldmzJlipCXlydoNBqhQ4cOwpQpU4TDhw9HelntzldffSUAaPIxa9YsQRDYUfiFCxcKOTk5glarFS699FLh4MGDkV10O9DS991qtQqXX365kJWVJajVaqGwsFC46aabhIqKikgvu83z9T0HILzyyiuex9hsNuGvf/2rkJaWJsTHxwsTJ04UysvLI7fodqC17/uJEyeECy+8UEhPTxe0Wq3QpUsX4Z577hGMRmNY1qf4Y5GEEEIIITGDaoAIIYQQEnMoACKEEEJIzKEAiBBCCCExhwIgQgghhMQcCoAIIYQQEnMoACKEEEJIzKEAiBBCCCExhwIgQki7MGrUKNxxxx2RXkaLioqKsGLFikgvgxACCoAIIX6aPXs2FAoFFAoF1Go1iouLce+994LjuEgvzS+vvvoqFAoFxo4d63W/wWCAQqHAtm3bIrMwQkhEUABECPHb2LFjUV5ejt9//x1PP/00XnzxRSxevDjSy/JbXFwcvvjiC3z11VeRXops7HZ7pJdASJtEARAhxG9arRa5ubkoKCjAhAkTMHr0aHz++eeez/M8j9tuuw3Z2dnQ6XQYOXIkfvjhB8/nX331VaSmpnpdc+PGjVAoFJ4/L1myBAMGDMAbb7yBoqIipKSkYOrUqTCbzZ7HWCwWzJw5E4mJicjLy8OTTz7p1/oTEhJwww034P7772/2Mdu2bYNCoYDBYPDct3fvXigUChw7dszr6/jkk0/QvXt3xMfH409/+hOsVitee+01FBUVIS0tDbfddhtcLpfX9c1mM6ZNm4aEhAR06NABK1eu9Pq8wWDAjTfeiKysLCQnJ+OSSy7Bzz//3OT78/LLL6O4uBg6nc6vr50Q4o0CIEJIQPbv348dO3ZAo9F47rv33nvx/vvv47XXXsOePXs8k8zPnTsn6dpHjhzBxo0b8cknn+CTTz7B9u3b8dhjj3k+f88992D79u348MMP8dlnn2Hbtm3Ys2ePX9desmQJfvnlF7z33nuS1tSY1WrFs88+i3feeQdbtmzBtm3bMHHiRGzevBmbN2/GG2+8gRdffLHJ6zzxxBPo378/fvrpJ9x///24/fbbvYLIyZMno6qqCv/5z3+we/duDBo0CJdeeqnX9/Dw4cN4//33sWHDBuzduzeor4OQmBWWkauEkDZv1qxZgkqlEhISEgStVisAEJRKpfDee+8JgiAIdXV1glqtFt566y3Pc+x2u5Cfny8sX75cEARBeOWVV4SUlBSv637wwQdCw3+KFi9eLMTHxwsmk8lz3z333COUlpYKgiAIZrNZ0Gg0wrvvvuv5fE1NjaDX64Xbb7+92fU3fO37779f6Natm+BwOITa2loBgPDVV18JglA/sb22ttbz3J9++kkAIBw9etRzLQDC4cOHPY+55ZZbhPj4eMFsNnvuGzNmjHDLLbd4/lxYWCiMHTvWa11TpkwRxo0bJwiCIPz3v/8VkpOTBY7jvB5TUlIivPjii57vj1qtFqqqqpr9WgkhraMMECHEbxdffDH27t2L77//HrNmzcKcOXMwadIkACxr43A4MGLECM/j1Wo1hg4dil9//VXS6xQVFSEpKcnz57y8PFRVVXlex263o7S01PP59PR0dO/e3e/r33fffaiursaaNWskrauh+Ph4lJSUeP6ck5ODoqIiJCYmet0nrls0fPjwJn8Wvz8///wz6urqkJGRgcTERM/H0aNHceTIEc9zCgsLkZWVFfDaCSFAXKQXQAhpOxISEtClSxcAwJo1a9C/f3+sXr0ac+fO9ev5SqUSgiB43edwOJo8Tq1We/1ZoVDA7XYHuOqmUlNTsWDBAixduhRXXXVVkzUC8Fqnv2sMdt11dXXIy8vzeSKtYe1UQkKC39ckhPhGGSBCSECUSiUeeOABPPTQQ7DZbCgpKYFGo8G3337reYzD4cAPP/yAXr16AQCysrJgNpthsVg8j5Faw1JSUgK1Wo3vv//ec19tbS1+++03Sdf529/+BqVSiWeeecbrfjGzUl5eHvAaW/Ldd981+XPPnj0BAIMGDUJFRQXi4uLQpUsXr4/MzEzZ1kAIoQCIEBKEyZMnQ6VSYeXKlUhISMC8efNwzz33YMuWLfi///s/3HTTTbBarZ4MUWlpKeLj4/HAAw/gyJEjePvtt/Hqq69Kes3ExETMnTsX99xzD7788kvs378fs2fP9mRu/KXT6bB06VI8++yzXvd36dIFBQUFWLJkCQ4dOoRNmzb5fcrMH99++y2WL1+O3377DStXrsT69etx++23AwBGjx6N4cOHY8KECfjss89w7Ngx7NixAw8++CB+/PFH2dZACKEAiBAShLi4OMyfPx/Lly+HxWLBY489hkmTJuH666/HoEGDcPjwYXz66adIS0sDwGp13nzzTWzevBl9+/bF2rVrsWTJEsmv+8QTT+CCCy7A+PHjMXr0aIwcORKDBw+WfJ1Zs2ahc+fOXvep1WqsXbsWBw4cQL9+/fD444/jkUcekXzt5tx111348ccfMXDgQDzyyCN46qmnMGbMGABsy2zz5s248MILMWfOHHTr1g1Tp07F8ePHkZOTI9saCCGAQmi8IU8IIYQQ0s5RBogQQgghMYcCIEIIIYTEHAqACCGEEBJzKAAihBBCSMyhAIgQQgghMYcCIEIIIYTEHAqACCGEEBJzKAAihBBCSMyhAIgQQgghMYcCIEIIIYTEHAqACCGEEBJzKAAihBBCSMz5fw6Wy5OiCm2MAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "al_acc = np.load('activelearn_acc.npy')\n",
        "rand_acc = np.load('random_acc.npy')\n",
        "\n",
        "rand_acc_std = compute_std_dev(rand_acc)\n",
        "al_acc_std = compute_std_dev(al_acc)\n",
        "\n",
        "plt.plot(range(1, al_acc.shape[1]+1), np.mean(al_acc, axis=0), label=\"active learning\", color='green')\n",
        "plt.fill_between(range(1, al_acc.shape[1]+1), al_acc_std[0], al_acc_std[1], alpha=0.3, color='green')\n",
        "\n",
        "plt.plot(range(1, rand_acc.shape[1]+1), np.mean(rand_acc, axis=0), label=\"random\", color='red')\n",
        "plt.fill_between(range(1, rand_acc.shape[1]+1), rand_acc_std[0], rand_acc_std[1], alpha=0.1, color='red')\n",
        "\n",
        "plt.hlines(y=0.9, xmin=1.0, xmax=25.0, color='black', linestyle='dotted')\n",
        "plt.legend()\n",
        "plt.xlabel(\"Round Number\")\n",
        "plt.ylabel(\"Test Accuracy\")\n",
        "plt.title(\"ActiveLab vs Random Annotation Selection --- 5 Runs\")\n",
        "plt.savefig(\"al-results.png\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e7d562c9",
      "metadata": {
        "id": "e7d562c9"
      },
      "source": [
        "We see that choosing what data to annotate next has drastic effects on model performance. Active learning using ActiveLab consistently outperforms random selection by a significant margin at each round. For example, in round 4 with 275 total annotations in the training set, we obtain 91% accuracy via active learning vs. only 76% accuracy without a clever selection strategy of what to annotate. Overall, the resulting Transformer models fit on the dataset constructed via active learning have around **50%** of the error-rate, no matter the total labeling budget!\n",
        "\n",
        "**When labeling data for text classification, you should consider active learning with the re-labeling option to better account for imperfect annotators.**"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
